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(57) Abstract: A recording medium storing an AVChp 
structured by multiplexing video stream and s graphi c;? 
stream. The graphics stream represents a moving picture 
made of a plurality of pictures, and the graphics stream 
includes graphics data representing graphics to be com- 
bined with the pictures. The graphics stream also includes 
window information (WDS) that specifies a window for 
rendering the graphics, and that indicates a width, a height 
and a position of the window on a plane which is a plane 
memory of a reproduction apparatus that combines the 
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DESCRIPTION 

RECORDING MEDIUM, REPRODUCTION APPARATUS, RECORDING METHOD, 
REPRODUCING METHOD, PROGRAM, AND INTEGRATED CIRCUIT 

5 

Technical Field 

The present invention relates to a recording medium such 
as a BD-ROM, and a reproduction apparatus, and more specifically, 
to a technique of subtitling by reproducing a digital stream 
10 constituted by multiplexing a video stream and a graphics stream. 

Background Art 

Subtitling realized by rendering graphics streams is an 
important technique for allowing people in different linguistic 

15 areas to appreciate a film produced in a language other than their 
native languages. An example of a conventional technique of 
subtitling is a memory allocation scheme for a Pixel Buffer based 
on -he ETSI EN 30C 7 43 standard sei forth by European 
Telecommunications Standards Institute (ETSI) . The Pixel Buffer 

20 is a memory for temporarily scoring decompressed graphics, and 
a reproduction apparatus writes the graphics in the Pixel Buffer 
to a display memory called a Graphics Plane, and thus the graphics 
is displayed. In the memory allocation scheme, a definition of 
a Region is included in the Pixel Buffer, and a part of the ^ 

25 decompressed graphics that corresponds to the Region is written 
to the Graphics Plane. For example, when a subtitle "Goodbye. . . " 
is contained in the Pixel Buffer and a position and a size of the 
Region are defined so as to includes a part N 'Gc", then the part 
xx Go" is written to the Graphics Plane and displayed on the screen. 

30 Likewise, when the position and size of the Region are defined 

1 
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so as to includes a part -Good", then the part -Good" is displayed 
on the screen. 

By repeating of the defining of the Region and the writing 
to the Graphics Plane, the subtitle -Goodbye. . ." is displayed on 
the screen gradually, i.e., first "Go", next -Good", then "Goodbye", 
and finally the whole subtitle -Goodbye..." .is disoiayed. By 
rendering a subtitle in such a way, it is possible .to realize a 
wipe-in effect. 

The ST SI EN 30 C 7 4 3 standard , however, does not at ail cons ider 
to guarantee the sync between a graphics display and a picture 
display when a burden for writing to the Graphics Plane is high. 
The graphics written to the Graphics Plane is not compressec, and 
accordingly, the burden for writing to the Graphics Plane increases 
as a resolution of the graphics becomes higher. A size of the 
graphics to be written to the Graphics Plane is up to 2 Mbytes 
when rendering' the graphics in a resolution of 1920*1080, which 
is a proposed standard resolution for a 3D-R0K, and a higher 
bandwidth for a graphics data transfer from the Pixel Buffer to 
the Graphics Plane is necessary in order to render graphics as 
large as 2 Mbytes synchronously with the picture display. However, 
demanding a high bandwidth for the aata transfer to write the 
graphics to the Graphics Plane hinders an attempt of cost reduction 
in manufacturing the reproduction apparatus. It is possible to 
lower the necessary bandwidth in writing to the Graphics Plane 
by having the reproduction apparatus always perform a -reasonable 
write" , in which only a difference from a previous display is written 
to the Graphics Plane. However, demanding the reproduction 
apparatus to always perform the -reasonable write" restricts 
software applicable to the reproduction apparatus. 

As described in the above, the high burden for writing to 
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the Grachfcs Plane demands that reproduction apparatuses operate 
in the high bandwidth or perform the reasonable write, and as a 
result, restricts produce deve 1 cpment of reproduction 
apparatuses . 

Disclosure of the Invention 

An object of the present invention is to provide a recording 
medium with which graphics may be updated synchronously with a 
picture display even when an amount of data to be written to a 
Graphics Plane is large. 

Zn order to achieve the above con ecc , an example of the 
recordinq medium according to the present invention is a recording 
medium used for storing data, said recording medium comprising: 
a digital stream constituted by multiplexing a video stream and 
a graphics stream, wherein said videc stream represents a moving 
oicture made of a plurality of pictures, and the graphics stream 
includes : graphics data representing graphics to be combined with 
the pictures; and window information that specifies a window for 
reriderinc the graphics therein, the window information indicating 
a width, a height and a position of the window on a plane, the 
plane being a plane memory of a reproduction apparatus that combines 
the graphics with the pictures . 

By specifying a part of the Plane corresponding to each 
picture as the window for render ing the graphics , it is not necessary 
that the reproduction apparatus renders the graphics for an entire 
plane, and it is sufficient that the reproduction apparatus renders 
the graphics only in a limited size of window. Because it is not 
necessary to render the graphics outside the window in the plane, 
the load of software in the reproduction apparatus may be reduced. 

Further, by setting a size of the window so as to ensure 
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a sync display between the graphics and the picture, it beccrr.es 
•possible fcr a producer who performs authoring to guarantee the 
sync display in any kind of reproduction apparatus , ever, when uodare 
of :ne graphics is performed in a worst case. 

Moreover, by setting a position and a size of the window 
by the window information, it is possible to adjust the position 
and size of the window in the authoring, so that the subtitles 
are cut cf the way of pictures when viewing the screen. Therefore, 
the visibility of the graphics are maintained even when the picture 
on the screen changes as time passes, and thus it is possible to 
maintain the quality of a- film. 

The worst case in updating the graphics means a case in which 
the graphics is updated in a least efficient operation, i.e. all 
clear and re-drawing of the window. When setting the size of the 
window in order to prepare for the worst case, it is desirable 
that the above recording medium is such that the width and height 
of the window are set so that a size of the window is 1/x of the 
plane, the plane corresponding to a size of each picture and x 
being a real number based on a ratio between a windov; update rate 
and a picture display rate. 

By setting the window size in this manner, a bandwidth on 
the reproduction apparatus that is necessary for writing to the 
graphics plane is set to a fixed value. By structuring the 
reproduction apparatus, so as to satisfy this bandwidth, it is 
possible to realize the sync display between the graphics and the 
picture regardless of the software mounted to the reproduction 
apparatus . 

As described above, it is possible to present a minimum 
standard for a structure of the reproduction apparatus. As long 
as the transfer rate is set so as to satisfy the minimum standard, 
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a design cf the reproduction apparatus is at the discretion of 
developers. Therefore, it is possible to expand the possibility 
in development of the reproduction apparatus. 

5 Brief Description Of The Drawings 

FIG . 1 illustrates an example of use of a reccroinc medium 
according to the present invention. 

FIG. 2 illustrates a structure cf a 3D-R0M. 
FIG. 3 is a diagram schematically illustrating a structure 
10 of an AVCIip. 

FIG . 4 A illustrates a structure of a presentation graphics 
stream. 

FIG. 43 illustrates a PBS packet obtained after functional 

segments are converted. 
» 

15 FIG . 5 illustrates a Iccical structure that is made of various 

i 

kinds of functional segments. 

.FKJ.6 illustrates a relation between a display position of 
a subtitle and an Epoch. 

FIG.7A illustrates syntax to define a Graphics Object in 
20 an Objedt Definition Segment (CDS) . 

i 

F£G.7B illustrates syntax of a Palette Definition Segment 

(PCS) , 

•Fip.SA illustrates syntax of. a Window Definition Segment 

(WDS) ■ 

25 FTG . 8B illustrates syntax of a Presentation Composition 

Segment^ (PCS) . 

F^Gw9 illustrates an example of a description of a Display 
Set for ; subtitling . 

FIG. 10 illustrates an example of a description of the WDS 
30 and PCS in a DS1. 
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F_G.I1 illustrates an example of a descripnon of the PCS 
,:n a DS2 . 

FIG. 12 illustrates an example of a description of the PCS 
in a DS3 , 

FIG. 13 is an example of a description of a Display Set when 
Cut-In/Cut is performed, illustrating along a timeline. 

FIG. 14 is an example of a description of a Display Set when 
Fade-In/Out is performed, illustrating along a timeline. 

FIG. 15 is an example of a description of a Display Set when 
Scrolling is performed, illustrating along a timeline. 

FIG. 16 is an example of a description of a Display Set when 
wipe-In/Out is performed, illustrating along a timeline. 

FIG. 17 is a diagram comparing two cases: a window has four 
Graphics Objects,, and a window has two Graphics Ob:ects. 

FIG . 18 illustrates an example of an algorithm for calculating 
a decode duration. 

FIG. 19 is a flowchart of the algorithm, of FIG. 18. 

FIGs . 2 OA and B are flowcharts of tne algorithm of FIG. IB. 

FIG. 2 1A illustrates a case in which each window has an Obj ect 
Definition Segment . 

FIGS.21B and C are timing charts showing orders among 
numerals referred to in FIG. 18. 

FIG. 2 2A illustrates a case in which each window has two Object 
Definition Segments. 

FIGs.22B and C are timing charts showing orders among 
numerals referred to in FIG. lb. 

FIG . 23A describes a case in which each of two Windows includes 
an ODS. 

FIG. 233 illustrates a case in wnich a decode period (2) is 
longer than a total of a clearing period (1) and a write period 
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(31) . 

FIG.23C illustrates a case in which a cotal of the clearing 
period (1) and the write period (31: is longer than the decode 
period [2] . 

FIG. 24 illustrates shifts in rime of update' descrioed in 
an example in ihe present specification. 

FIG. 25A illustrates four Display Sets that are described 
sc as co perform the above explained update . 

FIG.25B is a t lining chare shewing settings of DTS and PTS 
of functional segments included in the four Display Sets, 

FIG. 2 6 illustrates an internal structure of a reproduction 
apparatus according to the present invention. 

FIG. 27 illustrates sizes of write races Rx, Rc, and Rd r 
Graphics plane 6, Coded Data Buffer 13, Ob j ecc Buffer 15, and 
Composition Buffer 16. 

FIG. 28 is a timing chart illustrating a pipeline .processing 
by the reproduction apparatus. 

FIG. 29 illustrates a timing chare in a pipeline processing 
of a case in which decoding of zhe CDS ends before clearing of 
the Graphics Plane is completed. 

FIG. 30 is a flowchart illustrating a process of a loading 
operation of a functional segment. 

FIG. 31 shows an example cf multiplexing. 

FIG. 32 illustrates a manner in which a DS10 is loaded to 
the Coded Daca Buffer 13. 

FIG. 33 illusira-es loading of a DS1, the DS10, and a DS20 
in 1 a normal reproduction. 

FIG. 34 illustrates loading of the DS1, DS10, and DS2C in 
the normal reproduction as shown in FIG. 33. 

FIG. 35 illustrates a flowchart showing a process performed 
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by the Graphical Controller 17. 

FIG. 36 illustrates a flowchart showing the process performed 
by the Graphical Controller 17. 

FIG. 37 illustrates a flowchart showing the process performed 
by the Graphical Controller 17. 

FIG. 38 illustrates a pipeline process of the reproduction 
apparatus based on the PTS of the PDS . 

FIG. 39 is a diagram describes a significance of the END in 
the pipeline process of the reproduction apparatus. 

FIG. 40 illustrates an internal structure of the reo reduction 
apparatus according to a second embodiment . 

FIG. 41 schematically illustrates an operation of reacting 
cut and writing to the Graphics Planes that constitute a double 
buffer. 

FIG . 42 is a flowchart illustrating the manufacturing process 
of the BD-ROK according to a third embodiment. 

Best Mode for Carrying Out the Invention 
[First Embodiment ] 

A First Embodiment of a recording medium according to the 
present invention is explained below. 

FIG . I illustrates an example of use of the recording medium. 
In the drawing, BD-ROM 100 is the recording medium according to 
the present invention. .The BD-ROM 100 is used for providing data 
of movie works to a Home Theatre System structured by a reproduction 
apparatus 200, a television 300, and a remote controller 400. 

The recording medium according to the present invention is 
manufactured by an improvement in an application layer of a BD-ROM. 
FIG. 2 illustrates a structure of the 3D-ROK . 

In the drawing , the BD-ROM is shown at a bottom of the drawing, 
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and a ::a-k on "he 3D-ROM is shown above the BD— ROM . Tne "rack 
is actually in a spiral shape on the disc, but shown in a line 
in the drawing. The track includes a lead-in area, a volume area, 
and a lead-out area. The volume area in this drawing has a physical 
layer, a file system layer, and an application layer. At a top 
of the drawing, an application format of the BD-ROM is illustrated 
using a directory structure. As illustrated in the drawing, the 
3D- ROM has a directory 3DMV under the root directory, and the BDMV 
directory contains a file for scoring an AVClip with an extension 
M2TS ;XXX . M2TS) , a file for stcring administrative info for the 
AVClip with an extension CLPI (XXX. CLPI) , and a file for defining 
a logical Play List (PI) for the AVClip with an extension K?LS 
(YYY.MPLS) . By forming the above application fornai, it is 
possible to manufacture the recording medium according to the 
present invention. In a case in which there are mere than one 
file for each kind, it is preferable to provide three directories 
named STREAK, CLIPINF, and FLAYLIST under the BDMV to store the 
files with the same extension in one directory. Specifically, 
it is desirable to store the files with the extension M2TS in the 
ST RE AX, tne files with the extension CLPI in the C1IPIXF, and the 
files witn the extension MPLS in the PLAYLIST. 

An explanation about the AVClip (XXX.H2TS) m tne aze'-e 
application format is given below. 

The AVClip (XXX.M2TS) is a digital stream in MPEG- TS format 
(TS is Transport Stream) obtained by multiplexing a video stream, 
at least one audio stream, and a presentation graphics stream. 
The video stream represents pictures of the film, the audio stream 
represents sound of the film, and the presentation graphics stream 
represents subtitles of the film. FIG. 3 is a diagram schematically 
illustrating a structure of the AVClip. 

9 
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The AVCiip (XXX.M2TS) is siruc;urec a following nanner. 
Each of the video stream made of plural viae frames (picture pjl, 
p j 2 , and p j 3 ) , and rhe audio stream, made of plural audio frames 
(top row of the drawing) are converted into a line of PES packets 
(second row of the drawing) , and then into a line of TS packers 
(third row of the drawing) . The presentation graphics stream 
(bottom row of the drawing) is converted into PBS packets (second 
to bottom row of the drawing) , and then into TS packets (third 
to bottom row of the drawing) . Three lines of PS packets are 
multiplexed, and thus the AVCiip (XXX.M2TS) is constituted. 

In the drawing , only one presentation graphics stream is 
multiplexed. However, in a case in which the 3D-R0M is compatible 
to plural languages, a presentation graphics stream for each 
language is multiplexed to constitute tne AVCiip . The AVCiip 
constituted in the above manner is divided into more than one extent, 
like ordinary computer files, and stored in areas in the 3D -ROM . 

Next, thepresentation graphics stream is explained. TIG . 4A 
illustrates a structure of the presentation graphics stream. A 
top row indicates th e TS packet line to be multiplexed to the AVClio . 
A second to the top row indicates t he PES packet line that const i tu t es 
a graphics stream.. The PES packet line is structured by retrieving 
payioads out of TS packets having a predetermined ?ID, and 
connecting the retrieved payioads. 

A third to the top row indicates the structure of the graphics 
stream. The graphics stream, is made of functional segments named 
a Presentation Composi tion Segment (PCS ) , a Wind ow Definitio n 
Segment {VJDS ) , a Pa lette Definition Segment (PD S) , an Objec t 
Definition Segment (ODS) , and an E ND of Display Set Segment J EKB) . 
Among the above functional segments, the PCS is called a screen 
composition segment, and the WDS, PDS, ODS, and END are called 

10 
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definition segme nts. "he Pr-S packer and each of the functional 

segments correspond one :o one, or one to plurality. In otner 

words, one functional segment is either recorded in the 3D-R0IYI 

after converted into one FES oackei , or after divided into fragments 

«- . — „ 

and converted into mere than one PES packer, . 



FIG. 43 illustrates the PES packet obtained by converting 
:he functional seanents. As shown in the drawing, the PES packet 



is made of a packet header and the payload, and the p ayload ; Ls 
a substantial body of a functional segment. The packet header 



includes a DTS and a ?TS corresponding to the functional segment. 
The DTS and PTS included in the packet header are hereinafter 
referred to as the DTS and PTS of the functional segment. 

The above described various kind of functional segments 
constitute a logical structure as illustrated in FIG. 5. FIG. 5 
illustrates the logical structure that is made of the various kinds 
of functional segments. In the drawing, a nop row illustrates 
Epochs, a middle row illus 



trates Display Sets ( DS) , and a bottom^^ 
ow illustrates the functional segments, 



Each of the DS shown in the middle row is a group of functional 
20 segments that compose graphics for one screen, among all of the 
plural functional segments that constitute the graphics stream. 
Broken lines in the drawing indicate the DS to which the functional 
segments in the bottom row belong, and show that a series of the 
functional segments of the PCS , WDS, PDS, ODS , and END constitute 
25 one DS . The reproduction apparatus is able to generate graphics 
for one screen by reading the functional segments that constitute 
the DS. 

The Epochs shown in the top row indicate time periods, and 
memory management is consecutive timewise along a timeline of the 
30 AVClip reproduction in one Epoch. One Epoch also represents a 

11 
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croup of caca zr.az is assigned to the same period of time. The 
memory referred :o here are the Graphics Plane that scores the 
graphics for one screen, and an Object Buffer chat stores 
decomoressed graphics data. The consecut i veness of the memory 
management means a flash of the Graphics Plane or of the Object 
Buffer does no: occur in the Epoch, and erasing and rendering of 
the graphics are only performed in a predetermined rectangular 
area on che Graphics Plane (the flash here indicates clearing of 
all contends of the scored data in a plane or a buffer) . A size 
and a position of che rectangular area are fixed during one Epoch. 
As long as the erasing and rendering of che graphics are only 
performed in che predetermined rectangular area on the Graphics 
Plane, a sync reproduction between the picture and che graphics 
is guaranteed. In other words, the Epoch is a unit in the 
reproducing timeline, and in this unit, the picture and che graphics 
are guaranteed to be reproduced synchronously, when moving che 
area, in which the graphics are erased and rendered, to a different 
position, it is necessary to define a point on the timeline to 
move che area, and a period after che point becomes a new Epoch. 
The sync reproduction is not guaranteed at a boarder between cwc 
Epochs . 

In viewing an actual film, one Epoch is a time period in 
which subtitles are displayed in the same rectangular area on che 
screen. FIG. 6 illustrates a relacion becween the position of the 
subtitles and che Epochs . In an example ill us traced by the drawing, 
the positions at which the five subtitles "Actually. . . ", "I was 
hiding", "my feelings.", "I always", and "loved you." are shown 
move according to the picture in che film. Specifically, the 
subtitles "Actually . . . ", "1 was hiding", and "my feelings . " appear 
at che bottom of the screen, while the subtitles "I always" and 

12 
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"icved you." are snovr ; a~ the cop of the screen. The posinon 
of the rectangular area r.oves in order chat the subtitles are out 
of the way of pictures when viev.tng the screen, considering 
visibility of the film. A time period during which the subtitles 
appear at the bottom is an Epoch 1, ana a subsequent time period 
during which the subtitles appear at the top is an Epoch 2. The 
Epochs I and 2 each have a different area in which the subtitles 
are rendered. The area in the Epoch 1 is a Window 1 positioned 
at the bottom cf the screen, and the area in the Epoch 2 is a. Window 
2 positioned at the top of the screen. The memory management is 
consecutive in each cf the Epochs 1 and 2 r and accordingly, rendering 
of the subtitles in the Windows 1 and 2 is synchronous with the 
pictures . 

Next, details about the Display Set (DS) are described. 

Broken lines hkl 1 and hki2 in FIG . 5 indicate which functional 
segment at the middle row belongs to which Epoch. A series of 
DS "Epoch Start" , "Acquisition Point" , and "Normal Case" constitute 
the Epoch at the top raw. The "Epoch Start", "Acquisition Point", 
and "Normal Case" are types of the DS, and an order between the 
"Acquisition Point" and "Normal Case" does not matter and either 
cf them may come first. 

The Epoch Start is a DS that has a display effect cf "new 
display", which indicates a start of a new Epoch. Because of this, 
the Epoch Start contains- all functional segments needed to display 
a new composition cf the screen. The Epoch Start is provided at 
a position which is a target of a skip operation cf the AVCiip, 
such as a chapter in a film. 

The Acquisition Point is a DS that has a display effect of 
"display refresh", ■and is identical in content used for rendering 
graphics with the Epoch Start which is a preceding DS. The 

13 
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Acquisition Point is no: provided at a starting point of the Epoch, 
but contains all functional segments needed to display the new 
composition of the screen. Therefore, it is possiole to display 
the graohics without fail when a skip operation to tne Acquisition 
Point is performed. Accordingly/ with the Acquisition Point, it 
is possible tc compose a screen in the middle of the Epoch. 

The Acquisition Point is provided at a position that could 
oe a target for the skip operation . An example of such a position 
is a position that could be specified when performing a time search . 
The time search is an operation in response to a user' s input of 
a time to start reproducing from a reproducing point corresponding 
to the time specif iec by the user. The time is specified roughly, 
such as by 10 minutes or by 10 seconds, and accordingly, points 
at which the reoroduct i on starts are provided at such as a 10 minute 
interval, or a 10 second interval. By providing the Acquisition 
Point at the points at which the reproduction may start, it is 
possible to perform reproduction smoothly after the time search. 

The Normal Case is a DS that has a display effect of ''display 
uodate" , and contains only elements that are different from the 
oreceding composition of the screen. Specifically, when 
subtitles in a DSv is the same as subtitles in a DSu but the screen 
is displayed differently in the DSv and DSu, the DSv is provided 
so as to include only the PCS and makes the DSv the Normal Case. 
By this, it does not necessary to provide an CDS with the same 
content as the content of the ODS in the preceding DS , and a data 
size in the BD-ROM may be reduced. On the other hand, because 
the DS as the Normal Case contains only the difference, it is not 
possible to compose the screen using the Normal Case alone. 

Details of the Definition Segments (ODS, WDS, and PDS) are 
explained below. The Object Definition Segment (ODS) is a 
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functional segment that defines tne Graphics Object. An 
explanation of the Graphics Object is given first . A selling point 
of tne AVClip recorded in the BD-ROM is its resolution as high 
as hi -vis ion , and therefore the resolution for the Graphics Object 
is set at 1920*1080 pixels. 3ecause of the high resolution of 
1920*1080 pixels, it is possible to display a specific character 
style for the subtitles clearly on the screen. As for colors of 
the subtitles, a bit length of an index value for each pixel (Color 
Difference Red Cr, Color Difference Blue Cb, luminance Y , and 
Transparency T) is 8 bits, and thus it is possible to cncse any 
25 6 colors out of full color (16, 777,216 colors) for the subtitles. 
The subtitles realized by the Graphics Object are rendered by 
positioning texts on a transparent background. 

Syntax of ODS to define the Graphics Obj ect is shown in FIG . 7A . 
The ODS is made of segment_cype indicating that the segment is 
the CDS, segment_length indicating a data length of the ODS, 
ob~ ect id uniquely identifying the Graphics Ob j ect corresponding 
to the ODS in the Epoch, obj ect_ver s ion_n umber indicating a version 
of the CDS within the Epoch, last_insequence__f la g , and 
object data_f ragmen t which is a consecutive sequence of bytes 
corresponding to a part or all of Graphics Object. 

The obj ect_id is for uniquely identifying the Graphics Obj ecu 
corresponding to the ODS in the Epoch. The Epoch of the graphics 
stream contains more than one ODS having the same ID. The ODS 
having the same ID also have the same width and height, and are 
assigned with a common area in the Object Buffer. After one of 
the ODS having the same ID is read in the common area, the read 
ODS is overwritten by a subsequent ODS having the same ID. By 
overwriting the ODS that is read to the Object Buffer by the 
subsequent ODS having the same ID as the reproduction of the vide 
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stream proceeds, the graphics by the ODS is updated accordingly, 
A size constraint that the width and height of the Graphics Deject 
having the same ID should be the same is applied only during one 
Epoch, and the Graphics Objects in different Epochs may have 
different sizes. 

Explanations about las t_sequence__f lag and oo j ecc_ 
da ta_f ragment are given next. In some cases, it is not pcssicie 
to store the decompressed graphics that constitutes the subtitle 
in one OCS cue to a payioad constraint of the PES packer . In such 
cases , the graphics is spiitintoa seriesof consecutive f ragment s , 
and one fragment is set to the obj ect_daca_f ragment . When one 
Graphics Ob j ect is stored as more than one fragment, every fragment 
except a las: fragment has the same size. The last fragment is 
less than or equal to the size of previous fragments. The ODS 
carrying the fragments appear in the same sequential order in the 
DS, with an end of the sequence indicated by the ODS having the 
Iast_sequence_f lag . Although the above described syntax of the 
CDS is based on a premise that the fragments are stacked in from 
the preceding PES, the fragments may be stacked so that each FES 
contains a blank part. 

Next, the Palette Definition Segment (PDS) is explained. 
The PDS is used to define a palette for a color conversion . TIG . 73 
shovjs syntax of the PDS . ■ The PDS is made of segment_type indicating 
that the segment is the PDS, segmental ength indicating a data length 
of the PDS, palette__id uniquely identifying the palette contained 
in the PDS , palette_ version__number indicating a version of the 
PDS within the Epoch, and paiett e__entry\_id specifying an entry 
number of the palette. The paiette__entry_id indicates the Color 
Difference Red (Cr__value) , the Color Difference Blue (Cb_value) , 
Luminance CY_vaiue.) , and Transparency (T_value) . 
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Next, an explanation about the Window Definition Segment 
(WDS ; is given below.' 

The WDS is usee to define the rectangular area on the Graphics 
Plane. As described in the above, "he memory management is 
sequential only when erasing and rendering is performed within ' 
a certain area on the Graphics Plane. The area on the Graphics 
Plane is def inedby the WDS and called "Window" . FIG . 8A illustrates 
syntax of the WDS. As sho^r. by the drawing, the WDS is maoe of 
segment type indicating that the segment is the WDS, segment_lengtn 
indicating a data length of the WDS , window_id uniquely identifying 
the Window on the Graphics Plane, wrndow_horizontai_positior. 
specifying a horizontal address of a top left pixel of the Window 
on the Graphics Plane, windcw_vert ical_positicn specifying a 
. vertical address of the top left pixel of the Window on the Graphics 
Plane, window_width specifying a width of the Window on the Graphics 
Plane, and wincow_height specifying a height of the Window on the 

Graphics Plane. 

Ranges of values that the window_horizontal_position , 
windowjsrerticaljpcsitiori, window_yidth, and window_height may 
take are explained below. A coordinate system for those values 
is within an area on the Graphics Planed and whose size is i-ndipa'ted 
twc-dimensionally by the window__neight for a height and the 
window_width for a width. 

The window hori zontai_pcs it ion specifies the horizontal 
address of the top left pixel of the Window on the Graphics Plane, 
and is within a range of C to ( wiridow_width) -1 . Also, the 
window vertical_position specifies the vertical address of the 
top left pixel of the Window on the Graphics Plane, and is within 
a range of C to ( window_height ) -1 . 

Tne window width specifies the width of the Window on the 

17 



8/28/2007, EAST Version: 2.1.0.14 



WO 2004/098193 



PCT/JP2004/006074 



Graphics Plane. The specified width falls within a range of 1 
to { videc__width ) ~ ( window_hori zontal_j?osition) . Further, the 
wir_dow_height specifies the height of the Window on the Graphics 
Plane, and the specified height xs within a ranee of 1 to 
( vi dec_h eight ; - ( window_vertica imposition) > 

The position and size of the Window on the Graphics Plane 
for each Epoch are defined by the wir.dow_horizontal pos i t ion, 
window_vert ical_position, windcw_width, and window_height . 
Accordingly, it is possible to adjust the position and size of 
the Window at authoring, so that the Window in one Epoch appears 
at the position that does not come in the way of the picture when 
viewing the film. 3y this , the visibility of the subtitles becorr.es 
higher. Because the WDS is defined for each Epoch, it is possible 
to adjust the position of the Window according to the picture, 
even if the picture changes in the course of tine. As a result, 
the quality of the film is maintained as high as in a case where 
the subtitles are incorporated in the main body of the film. 

Next, the End of Display Set Segment (END) is explained. 
The END provides an indication that a transmission of the DS is 
completed. The End is inserted into a stream immediately after 
a last ODS in one DS . The End is made of segment__t ype indicating 
that the segment is the END and segmental ength indicating a data 
length of the END. The END does not include any other element 
that requires a further explanation. 

Next, an explanation about the Presentation Composition 
Segment (PCS) is given below. 

The PCS is a functional segment that is used for composing 
an interactive display. FIG. 8B illustrate syntax of the PCS. As 
shown in the drawing, the PCS is made of segment__type , 
segment_iength , compos it ion__number , composit ion__stat e , 
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palette_update_flag, paiette_id, and window information i-m. 

The compcsitior._number identifies the Graph-ics Update in 
the DS by values in a range of 0 to 15. If the Graphics Update 
exists between the neac of the Epoch and the PCS, the 
composition_number is incremented every time' the Graphics Update 
occurs . 

The composition_state indicates the type of the DS in which 
the PCS is contained, Normal Case, Acquisition Point, or Epoch 
Start . 

■ The palet-e_update__flag indicates that the PCS describes 
a Palette only Display Update. The Palette only Display Update 
indicates that only the palette is updated from an immediately 
previous palette. The palette_update_f lag field is set to "1", 
if the Palette only Display Update is performed. 

The oaiette id identifies the palette to be used in the 
Palette only Display Update. 

The window information 1-ir, indicate how to control each 
Window in the DS to which the PCS belong. A broken line wdl in 
FIG.8B is to detail an internal syntax for window information i. 
The window information i is made of objected, winriow_id, 
object_crcppec_flag, ob j ect__hcri zone al_position , an 

object_verticaI_pcs±tion, and cropping_rectangie information 
n . 

j-v The object_id identifies the CDS in a Window corresponding 
to the window information i. 

The window_id identifies the Window to which the Graphics 
Object is allocated in the PCS. Up to two Graphics Objects may 
be assigned to one Window. 

The object_cropped_flac is used to switch between display 
and no-display of a cropped Graphics Object in the Object Buffer. 
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7:her. che ob j ect_cr opoea_f lag is set to N> 1", che croppec Graphics 
Object is displayed in the Object Buffer, and if set tc "0", the 
Graphics Object is r.c: displayed. 

The obj ecr_hori zontal_posit ion specifies a Horizontal 
address of a top lefc pixel of the Graphics Object in che Graphics 
Plane . 

The obj ecc__verc i ca Imposition specifies a vertical address 
of che top left oixel of the Graphics Object ir. tne Graphics Plane. 

The crooping rectangle information i-n are elements used 
wher. che obj ect_cro??ed_£lag is sec to " 1 " . A broken line wd2 
is co detail an internal syntax for croppir.g__recran.gle information 
i. As shown by che broken line wd2 , the cropping__rect angle 
information i is made of four fields, 

ob jecc_cropping_hori z onca Imposition , cb j ect_cropping__ 

vertical position, ob j ect__cropping__widrh , and object__ 
c r cpp i n g_h eight . 

The ob j ecr_cropping_hor izoncal_pos i t ion specifies a 
horizontal address of a top left corner of a cropping rectangle 
co be used during rendering of the Graphics Obj ecz in the Graphics 
Plane. The cropping rectangle is a cropping frame chat is used 
to specify and crop a part of the Graphics Objecr, and corresponds 
to Region in the ETS1 EN 300 743 standard. 

The obj ect__crcpping_verc ical_posi tion specifies a vertical 
address of che top left corner of the cropping rectangle to be 
used during rendering of the Graphics Obj ect in tne Graphics Plane . 

The obj ect__cropping__width specifies a width of the cropping 
reccangle . 

The ob ject_cropping_ height specifies a height of the 
cropping rectangle. 

A specific example of the PCS is decaiied below. In the 
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example, the subtitles "Actually...", "I was. hiding", and "my 
feelings." as shown in FIG. 6 appear gradually by writing to the 
Graphics Plane 3 times as the picture proceeds . FIG. 9 is an example 
of description for realizing such a subtitle display. An Epoch 
in -he drawing includes a DS1 {Epoch Start) , a DS2 {Normal Case), 
and- a DS3 (Normal Case) . The DS 1 contains a WDS for specifying 
the Window in which the subtitles are displayed, an ODS for 
specifying the line " Actually ... I was hiding my feelings . ", and 
a first PCS. The DS2 contains a second PCS , and the -S3 contains 
a. third PCS . 

TIGs . 10-12 illustrate examples cf the WDS and PCS contained 
in the DS . FIG. 10 shows an example of the PCS in the DS1 . 

In FIG. 10, the' windowjnori zontai_pos it ion and the 
windov/__vertical_position of the WDS are indicated oy a LP I, a 
position of the uop left pixel of the Window on the Graphics Plane. 
The window_width and window_height indicate che widih and height 
of -he window, respectively. 

In FIG . 10 , the obj ect_cropp±ng_hcrizontai_posit ion and 
ob: ect__cropping_vertical_posicion indicate a reference poin~ SI1 
of the crooning rectangle in the coordinate syscem in which an 
origin is the top left pixel of the Graphics Object. The cropping 
rectangle is an area having the width from rhe ST uc the 
object_croppir.g_width, and the height from the ST to the 
cbject_croppingJieight -(a rectangle shown by a heavy-line frame) . 
The cropped Graphics Object is positioned within a rectangle shown 
by a broken-line frame cpl, with a reference poinn in the coordinate 
system with an origin at the object_horizontal_posi-tion and 
obj ect_verticaI_position (the top left pixel of the Graphics 
Obj ect) in the Graphics Plane . By this, the subtitle "Actually. . . " 
is written to the Window on the Graphics Plane, and then composed 
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with the movie picture and displayed on the screen. 

FIG . 11 shows an example of the PCS in the D52. The WDS in 
the DS2 is not explained, because the WDS in the DS2 is the same 
as the WDS in the DS1 . A description of the cropping information 
in the DS2 is different from the description of the cropping 
information shown in FIG . 1G . 

In FIG ► 11, the cb j ect__cropp±nc_hor i zont al_jpositior, and 
ob j ect_cropping__vert ical_posi tion in the cropping information 
indicate a top left pixel of the subtitle "I was hiding" out of 
"Actually. . . I was hiding my feelings." in the Object Buffer. 
The ob ject_cropp±ng_vidth and ob j ect_cropping__neight indicates 
a width and a heigh". of a rectangle containing the subtitle "I 
was hiding". 3y this, the subtitle "I was hiding" is written to 
the Window on the Graphics Plane, and then composed with the movie 
picture and displayed on the screen. 

FIG . 12 shows an example of the PCS in the D33. The WDS in 
the DS3- is not explained, because the WDS in the DS3 is the same 
as the WDS in the DS1 . A description of the cropping information 
in the DS3 is different from the description of the cropping 
information shown in FIG. 10. 

In FIG. 12, the ob j ect__cropping_hori zor.tal_pcsiticn and 
ob j ect_cropping_vert ical_posit ion in the cropping information 
indicate a top left pixel of the subtitle "my feelings." out of 
"Actually... I was hiding my feelings." in the Object Buffer. 
The obj ect_cropping__width and ob j ect_cropping_height indicates 
a width and a height of a rectangle containing the subtitle "my 
feelings.". By this, the subtitle u my feelings." is written to 
the Window on the Graphics Plane, and then composed with the movie 
picture and displayed on the screen. 

3y describing the DS1, DS2, and DS3 as explained above, it 
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is oosstble to achieve an effect of displaying the subtitles or. 
tee screen. It is also possible to achieve other kinds of effect, 
and description protocols for realizing ether effects are explained 
below . 

First, a description protocol for a Cut-in/Out effect is 
exoiatned. FIG. 13 shows an example of the description cf the DS 
when Cut-In/Out is performed, illustrating along a timeline. 

In the drawing, x and y in Window (x, y, u, v) respectively 
indicate values of the window__vertical_positior. and 
window horizcr.tal_positio-, and u and v respectively indicate 
values of the window_width and window__h eight . Also in the drawing, 
a andb in Cropping Rectangle (a,b,.c, d) respectively indicate values 
of the cb j ect_c r opp in g_vertica Imposition and 

object crocping_horizontal_posicion, and c and d indicate values 
of the ob j ect__cropping__width and ob j ect_cropp: nc__h eight , 
respectively . Display Sets DS11, DS12, and DS13 are an points 
til, c!2 , and t!3 on the reproduction timeline in the drawing . 

The DSli at the point til includes a PCS#C in which the 
compos it ion_st ate is "Epoch Start 7 ' and the ob j ect__crcpped_f lag 
is XV C" (no_cropping_rectangle_visible) , a WDS#C having a statement 
for a Window in a width 700 * height 500 at (100, 100) in the Graphics 
Plane, a PDS#0, an OD3#0 indicating a subtitle ''Credits:", and 
an END. 

The DS12 at the point tl2 includes a PCS#1 whose 
composition_state is "Normal Case'' and indicating a crop operation 
of the Graphics Object to be in a 600*400 size from (0,0) in the 
Object Buffer { cropping_rectangLe.#0 ( 0 , C , 600 , 400 ) ; , and 
positioning the cropped Graphics Object at the coordinates (0,0)' 
in the Graphics Plane (on Window#0 (0, 0) ) . 

The DS13 at the point tl3 includes a ?CS#2 whose 
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ccmoosit ion s~ate is "Ncrnal Case" and in which z'r.e 
ob j ect croppec_f lac :ssei:o u C" sc as to erase the cropped Graphics 
Objec- ;no_crcpping_rectangle_visibie ; . 

With zhe above explained Display Sets, the subzitle 
u Cr edits : " is nc-di splay at ~he tl 1 , appears at the 1 12 , then oecon.es 
no-display at zhe tl 3 again, and zhe Cut-In/Cut-Out effect is 
reali zed , 

Secondly, a description protocol for a Fade-In/Out effecc 
is explained. FIG. 14 shows an example of the description of the 
LfS when Fade-in/Out is performed, illustrating along a timeline. 
Display Sens DS21, DS22, DS23, and DS24 are az points t21, z22, 
t2 3 , and t24 on the reproduction timeline in the drawing. 

The DS21 at ihe point 1 2 1 includes a PCS# C whose 
composi-icn_state is "Epoch Start" and indicating the crop 
opera-ion of the Graphics Object to be in a 600><40C size f rorr. (0, 3) 
in the Ooject Buffer (crop?ing_rectangle#0 ( 0 , 0 , 600 , 4 00 ) ; , and 
positioning the cropped Graphics Object at the coordinates (0,0) 
in the Graphics Plane (on window* 0 (0,0) ) , a WDS#0 having a statement 
for a Window in a width 700 « height 50 C at (100,100) in the Graphics 
Plane, a ?DS#0, an OD3#0 indicating a subtitle "Fin", and an FIND. 

The DS22 at the point t22 includes a PCS#1 whose 
compos it ion_st ate is "Normal Case'', and a ?DS#1. The ?DS#1 
indicates the same level of Cr and Co as the ?DS#0, but a luminance 
indicated by zhe PDS#i is higher than uhe luminance in the ?DS#0. 

The DS23 at the point t23 includes a ?CS#2 whose 
compositicn_scate is "Normal Case", a?DS#2, and an END. The ?DS#2 
indicates the same level of Cr and Cb as the PDS#1, but the luminance 
indicated by the PDS#2 is lower than the luminance in the PDS#1. 

The DS24 at the point t24 includes a PCS whose 
composition_state is "Normal Case" and the ob j ect__cropped_f lag 
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is "C" (r.o_crcppir.g_rectar.gie_visible; , and an END. 

Each DS specifies a different PCS from a preceding DS, and 
accordingly, the luminance of the Graphics Object "hat is rendered 
with more than one PCS in one Epoch becomes gradually high, or 
lev; . 3y this, ic is possible to realize the effect of Fade -In/Out . 

Nexz, a description proceed for a Scrolling is explained. 
FIG. 15 shows an example of the description of the DS when Scrolling 
is performed, illustrating along a timeline. Display Sets DS31, 
DS32, DS33, and CS34 are at points 1 31 , t32 , 1 33 , and u34 on the 
reproduction cimeline in the drawing. 

The DS31 au the point t31 includes a ?CS#0 whose 
compos it ion__sc ace us set to "Epoch Star:" and obj ect_crcpped_f lag 
is %s 0 " (no_cropping__rectangIe__visible) , a WDS#C having a statement 
for a Window in a widuh 7 CO * height 500 az (103, 100) in the Graphics 
Plane, a ?DS#0, an 0DS#C indicating a subtitle "Credics: Company", 
and an END. 

The DS32 at the point t32 includes a ?CS#i whose 
composition state is "Normal Case" and indicating the- crop 
operation of che Graphics Object to be in a 600*4 00 size from (0,0) 
in the Obj ec: Buffer ( cropping_rectangie#G ( C , 0 , 600 , 4 00 } } , and 
positioning the cropped Graphics Object at the coordinates !0,C) 
in the Graphics Plane (on WindowSC (0, 0) ) . An area of the 600x400 
size from (0,0) in che Object Buffer includes a pare "Credits:" 
of the subtitle "Credits: Company" shown in two lines, and thus 
the part ''Credits:" appears on the Graphics Plane. 

The DS33 at the point 1 33 includes a' PCS#2 whose 
composition state is "Normal Case" and indicating the crop 
operation of the Graphics Object to be in a 60 0 * 4 0 0 size from (0, 100 ) 
in the Object Buffer (cropping_rectangle#0 (0,100, 600,400)), and 
positioning the cropped Graphics Object at che coordinates (0,0) 
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in ~he Graphics Plane (on Svindow#0 ( C , C ; ') . Che area of the 600* 4 30 
size from ( 0 , ICC ; in the Obj ect Buffer includes the part "Credits : " 
and a part "Company" of the subtirle ''Credits: Company" shown in 
two lines, and thus the parts "Credits:" and "Company" appear in 
two lines on :ne Graphics Plane , 

The DS34 at the point t34 includes a PCS# 3 whose 
composition_s ta i.e is "Normal Case" and indicating :ne crop 
operation of the Graphics Objecr to be in a 600*400 size from (0, 2CC) 
in the Object 3uffer ( cropping_rectangle# C (0,200, 600, 400)), ana 
positioning the cropped Graphics Object at the coordinates (0,0 
in the Graphics Plane (on Window* 0 (CO) . The area of the 600*400 
size from (0, 200; in the Obj ecu Buffer includes the part "Company" 
of the subtitle "Credits: Company" shown in two lines, and thus 
the part "Company" appears on the Graphics Plane. By the above 
PCS description., it is possible to scroll down the subtitle in 
two lines . 

Finally, a description protocol for a &vipe — In/Out effect 
is explained. FIG. 16 shows an example of the- description of the 
DS when wipe-In/Out is performed, illustrating along a timeline. 
Display Sets 2321, DS22, DS23, and D32 4 are at points t21, c22, 
t23 , and t24 on ihe reproduction timeline in the drawing. 

The D351 at the point t51 includes a PCS#0 whose 
compcsition_state is set to "Epoch Start" and the 
obj ect_cropped_f lag is* "0" (no_crcpprng_rectangle_visible) , a 
WDS#0 having a statement for a Window in a width 7 00 * height 500 
at (100,100) in the Graphics Plane, a ?DS#0, an ODS#0 indicating 
a subtitle "Fin", and an END . 

The DS52 at the point t52 includes a ?CS#1 whose 
composition^state is "Normal Case" and indicating the crop 
operation of the Graphics Object to be in a 6C0><400 size from (0,0) 
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m the Object Buffer ( cropping_rectang!e#C ( 0 , 0 , 600 , 400 ) } , and 
positioning the cropped Graphics Object at the coordinates (0,0) 
in tne Graphics Plane (on Window#0 (CO) ) . An area of the 600x400 
size from (0,0) in the Ob jeer Buffer includes the subtitle "Fin", 
and thus the subtitle "Fin" appears on tne Graphics Plane. 

The DS53 at "he point t53 includes a ?CS#2 whose 
composition^state is "Normal Case" ana indicating the crop 
operation of the Graphics Obj ect to be in a 4 00*400 size from (2C3, 0) 
ir. the Object Buffer (cropping_rect:angle#0 [200,0,400, 400)), and 
positioning the cropped Graphics Object at the coordinates (20C, 0) 
in the Graphics Plane (on Window#0 ( 200 , 0 ) ) . 3y this, an area 
indicated by coordinates (200,0) and (400,400) in the Window 
oecomes a display area, and an area indicated by coordinates (0,0) 
and (199,400) becomes a no-display area. 

The DS54 at the point t54 includes a ?CS£3 whose 
compcsition__state is "Normal Case" and indicating the crop 
operation of the Graphics Obj ect to be in a 200*400 size from (400,0) 
in the Obiect Bu ffer (crop?ing_rectangle#0 (400,0,200, 400)), and 
positioning the cropped Graphics Object at the coordinates (400,0) 
in the Graphics Plane (on Window#0 (400, 0 > > . 3y this, an area 
indicatedby coordinates (0,0) and (399,400) becomes the no-display 
area . 

By this, as the no-display area becomes larger, the display 
area becomes smaller, an-d thus the Wipe-In/Out effect is realized. 

As described above, various effects such as Cut-In/Out, 
Fade-In/Out, Wipe-In/Out, and Scrolling may be realized using 
corresponding scripts, and therefore it is possible to make various 
arrangements in rendering the subtitles. 

Constraints for realizing the above effects are as follows . 
In order to realize the Scrolling effect, operations for clearing 
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and redrawing of ine Window becomes necessary . Taking the example 
of FIG .15, it i s necessary :o perform "window clear'' io erase the 
Graphics Object "Credits : " an the t 32 from the Graphics Plane, 
and then tc perform NV window redraw' 7 :c write a lower part of 
''Credits:" and an upper pari of "Company" to the Graphics Plane 
during an interval between the 1 32 and 1 33 . Given thai the interval 
is the same as an interval of video frames, a transfer rate between 
the Object Buffer and the Graphics Plane desirable for the Scrolling 
effect becomes an important point. 

Here, a constraint about how large the Window may be is looked 
into. An Rc is the transfer rate between the Object Buffer and 
the Graphics Plane. A worst scenario here is to perform both of 
the Window clear and Window redraw at the rate Rc . In this case, 
each of the window clear and Window redraw is required to be performed 
at a rate half of Rc ;Rc/2) . 

In order to make the window clear and Window redraw 
synchronized with a video frame, an equation oeiow is need tc be 
satisfied . 

Window size * Frame Rate =F Rc/2 

If the Frame Rate is 29.97, Rc is expressed by an equation 

below . 

Rc = Window size x 2 *29.97 

In rendering the subtitles, the Window size accounts for 
at least 25% to 33% of the- Graphics Plane . A total number of pixels 
in the Graphics Plane is 1920* 108 C , Taking that an index bit length 
per pixel is 8 bins, a total capacity of the Graphics Plane is 
2 Mbytes (±=1920*1080x8). 

Taking that the Window size is 1/4 of the total capacity 
of the Graphics Plane, the window size becomes 50C Kbytes (=2 Mbytes 
/A). By substituting this value to the above equation, Rc is 
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calculated to be 256 Mbps { = 500 Kbytes * 2x29.97). If the rate 
for the Window clear and Window redraw may be a half or a quarter 
of the frame rate, it is possible to double or quadruple the size 
of the Window even if the Rc is the same. 

Bv keeping the Window size 25% to 33% of the Graphics Plane 
and tiisclaying the subtitles at the transfer -rate of 256 Mbps, 
it is possible to maintain the sync display between the graphics 
and the movie picture, no matter what kind of display effect is 
to be realized. 

Next, the position, size, and area of the window are explained . 
As explained above, the position and area of the Window does not 
change in one Epoch. The pcsition and the size of the Window set 
to be the same during one Epoch because it is necessary to change 
a target write address of the Graphics Plane if the position and 
the size change, and changing the address causes an overhead that 
lowers the transfer rate from the Object Buffer to the Graphics 
Plane . 

A number of Graphics Objects per Window has a limitation. 
The limitation of the number is provided in order to reduce the 
overhead in transferring decoded Graphics Object. The overhead 
here is generated when setting the address of an edge of the Graphics 
Object, and the mere a number of edges, the more the overhead is 
generated. 

FIG. 17 shows examples in comparison, an example in which 
a Window has four Graphics Objects and another example in which 
a Window has two Graphics Objects. The number of the edges of 
the example with four Graphics Objects is twofold of the number 
of the edges of the example with two Graphics Objects. 

Without the limitation in the number of the Graphics Object, 
it becomes unknown how many overheads could be generated in 
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transferring the Graphics, and thus the load for one transfer 
increases and decreases drastically. On the other hand, when a 
maximum number of the Graphics Ob j ect in a Window is two , the transfer 
rate may be set taking up :o 4 overhead into account . Accordingly, 
5 Lz is easier to set the number of a minimum transfer rate. 

Next, an explanation about how the DS . having the PCS and 
CDS is assigned to the timeline of one AVClip. The Spoch is a 
period of time in which a memory management is consecutive along 
the reproduction cimeline. Since uhe Epoch is made of more than 

10 one DS, how to assign one DS to the reproduction cimeline of the 
AVClip is important. The reproduction timeline of the AVClip is 
i timeline for specifying timings for decoding and reproducing 
of each piece of picture data that constitute the video stream 
multiplexed uo the. AVClip. The decoding and reproducing timings 

15 on the reproduction timeline are expressed ac an accuracy of 90 
.KHz. A DTS and ?TS thai are attached zo the PCS and ODS in the 
DS indicate timings for a synchronic control on the reproduction 
timeline. The assigning of the Display Set no the reproduction 
cimeiine means performing the synchronic control using the DTS 

20 and PTS attached tc the PCS and ODS . 

First, how the synchronic control is performed using the 
DTS and PTS attached :o the ODS is explained below. 

The DTS indicates, a: the accuracy of 90 KHz, a time when 
one decoding of the ODS stares, and the PTS indicates a time when 
• 25 the decoding ends . 

The decoding of one ODS does not finish at once, and has 
a certain length of time. In response to a request for clearly 
indicating a start ing poinr and an ending point of a decode duration, 
the DTS and PTS of the ODS respectively indicate the times when 

30 the decoding starts and ends. 
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The value of the PTS indicates the deadline, and therefore 
it is necessary that the decoding of the CDS has to be completed 
by the time indicated by the ?TS and the decompressed Graphics 
Object is written to the Object Buffer on the reproduction 
apparatus . 

The decode starting time of any CDS j in a DSn is indicated 
by a DTS {DSn 'ODS j ) at the accuracy of 3C KHz. Adding a maximum 
length of the aecode duration to the DTS ( DSn [ODS] ) is the time 
when the decoding of the ODSj ends. 

When a size of the ODSj is "SIZE (DSn [ODSj ]) " and a decoding 
rate of the ODS is an "Rd", the maximum time required for decoding 
indicated by second is expressed in "SIZE- (DSn [ODSj ]) //Rd" . The 
symbol "//" indicates an operator for a division with rounding 
up after a decimal place. 

By converting the maximum time period into a number expressed 
at the accuracy of 90 KHz and adding to the DTS of the ODS;, the 
time when the decoding ends (90 KHz) indicated by the PTS is 
calculated . 

The PTS of the ODSj in the DSn is expressed in a following 
equation . 

PTS ( DSn [ODSj j ).= DTS (DSn [ODS j ] >+9C, 030x (SIZE ( DSn [ ODS j [ ) / /Rd) 

Further, it is necessary that a relation between two 
succeeding ODS, ODSj and ODSj +1, satisfies a following equation. 
PTS ! DSn [ODSj ] ) ^DTS (DSn-fODSj + H ) 

Next, settings of the DTS and PTS of the PCS are explained .. 
It is necessary that the PCS is loaded to the Object Buffe r 
on the reproduction apparatus before the decode starting time 
( DTS ( DSn [ODS 1 ] ) ) of a first ODS (ODSi) in the DSn, and before the 
time (PTS(DSn[PDSlJ ) ) when a first PDS (PDSl) in the DSn becoir.es 
effective. Accordingly, it is necessary that the DTS is set sc 
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as to satisfy following equations. 
DTS (DSn [PCS] ) ^DTS (DSn :CDS1] ) 
DTS (DSntPCS] ) ^?TS { DSn [ PDSI J ) 

Fur-cher, the PTS of the PCS in "he C3n is expressed in a 
following equation . 

PTS (DSn [PCS] ) >DTS { DSn [ PCS ] ) +decodedurat ion (DSn) 

The "decodeduration (DSn) " indicates a time duration for 
decoding all "he Graphics Objects used for updating PCS . The decode 
duration is no: a fixed value, but does not vary according to a 
status of the reproduction apparatus and a device or a software 
moumed to the reproduction apparatus. When the Object used for 
composing a. screen of a DSn. PCSn is a DSn . PCSn . OB J r _ j ] , the 
decodeduraticn (DSn) is affected by time (i ) needed for clearing 
the Window, decoct durations (ii) for decoding a DSn . PCSn . OBJ , 
and time (iii) needed for writing of the DSn . PCSn . OBJ . When the 
Rd and Ro are set, the decode_dura tier. (DSn) is always the same. 
Therefore, the PTS is calculated by calculating lengths of these 
durations in authoring. 

The calculation of the decode_curation is performed based 
on a program shown in FIG. 18 . FIGs . 19, 20A and 203 are flowcharts 
schematically showing algorithms of the program. An explanation 
about the calculation of the decode__curation is given below 
referring to these drawings. In the flowchart shown in PIG. 19, 
first, a PLAN E I K T I T I AL Z £ function is called (Step SI in FIG. 19). 
The PLANSINITIALZE function is used for calling a function for 
calculating a time period necessary to initialize the Graphics 
Plane for rendering the DS . In the Step SI in FIG .19, the function 
is called with arguments DSn, DSn . PCS . OBJ [ 0 ] , and deccde__dura tion . 

The following explains the PLAN E I N I T I AL Z E function in 
reference to FIG.20A, In the drawing, initial! ze_duratior. is a 
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variable indicating a reiurn value of the PLANE I NIT I AL ZE function . 

Step S2 m 71 G .20 is an if statement f cr switching operations 
depending on whether or not the page__staoe in the PCS in the DSn 
indicates the Epoch Start. If the page__state indicates the Epoch 
Start (DSn . PCS . page_state— epoch__starc , Step S2=Yes in FIG. 18) , 
a time period necessary tc clear the Graphics Plane is set to an 
initialize_durauion (Seep S3) . 

When the transfer rate Rc between the Object Buffer and the 
Graphics Plane is 256,000,000 as described in the above, and the 
total size of the Graphics Plane is set to video_width*video_height , 
the time period necessary to clear is 

x Videc_width*video_height//256, 000, 000" . When multiplied by 
90.000Hz so as to express at the time accuracy of PTS, the time 
period necessary to clear the Graphics Plane is 
"90, D00*videc_width*videc_height://2 56, 0 00, 000". This time 
period is added to the initial! ze_dur at i on. 

If the page_state does not indicate the Epoch Start (Step 
S2^No) , a time period necessary to clear Window [ij defined by the 
WDS is added to the initialize_dur anion for all Windows (Step 54) . 
'When the transfer rate Rc between the Object Buffer and the Graphics 
Plane is 256,000,000 as described in the above and a total size 
of Winodow;i; that belongs to the WDS is £SIZE ( WDS . WIN [ i ] ) , the 
time period necessary to clear is 

"£SIZE (WDS . WIN [i] ) //256, 000, 000" . When multiplied by 30 . OOCHz 
so as tc express at the time accuracy of PTS , the time period 
necessary tc clear the Windows that belong to the "WDS is 
"90, 000*£SIZE(WDS. WIN[i] )//256, 000,000". This time period is 
added to the initislize_duration, and the initialize_duraticn as 
a result is returned. The above is the PLANEINITIALZE function. 

SteD S3 in FIG. 19 for switching operations depending on 
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whether me number of the Graphics Goiects in the DSn is 2 or 1 
(if (DSn. PCS . num_cf_object==2 , if (DSn. PCS . num_o£ __ob j ect==I in 
FIG. 18) , and if the number is 1 ( Step S5 ) , a waiting time for decoding 
the Graphics Object is added tc the decode_duraticn (Step S6) . 
Calculation of the waiting time is performed by calling a WAIT 
function ( decode_durat ion +=WAIT (DSn, DS . PCS . OB-J [ 0 ] , 

decode duration) in FIG. 18) . The function is called using 
arguments set to DSn, DSn . PCS . OBJ [0 \ , decode_duration, and a return 
value is wai t_duration . 

FIG . 203 is a flowchart showing an operation of the WAIT 
function . 

In the flowchart, the decode_durat ion of an invoker is set 
as a cur rent_durac ion . An obj ect_de f ini~ ior.__ready_t ime is a 
variable set to the PTS of the Graphics Object of the DS . 

A current_time is a variable set to a total value of the 
current__duratior. and the DTS of the PCS _in the DSn . When the 
object def inition_ready__t ime is larger than the current__time (Yes 
to Step S7, if ( current__time <obj ect_defir.it ion _ready_ time ) ) , the 
wait_durat ion as the return value is set tc be a difference between 
the object def ini t ion_ready_tirne and the current _ t ime (Step S8, 
wait duration - = object __def initi on_ready__time - current_t ime ) . 
The decode_duration is set tc the time period that the return value 
of the WATT function added to the time period necessary for 
re-drawing the Window, (90, 003 * ( SIZE ( DSn . WDS > WIN [ 0 ]) ) // 
256, 000, 000) . 

The above explanation is for the case in which the number 
of the Graphics Object is one. In Step S5 in FIG. 5, it is judged 
if the number of the Graphics Objects is two. If the number of 
the Graphics Objects in the DSn is more than two 
(if (DSn. PCS .num__of_object==2) in FIG .18), the WAIT function is 
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called using OEJ[0] in the PCS as an argument, and add a return 
value to the deccde_durat ion [Step SIC) . 

In a succeeding Step Sll, it is judged if the Window tc which 
-he 03J[C] of the DSn belongs is the same as the Window to which 
the Graphics Object ;i; belongs ( if (DSn . OBJ [ c; . window_ic== 
DSn . PCS . OBJ [ 1 j . window_id) . If the window is the same , the WAIT 
function is called using OBJ[l] as an argument, and add a return 
value wait duration to the aecode_durat ion (Step S12) , and add 
the Lime necessary to redraw the Window to which 03J[C] belong 
; 90, COC - (SIZE ( DSn . WDS . OBJ [C] . window_id) ) / 72 56, 003, 0CC ) to the 
decode_duration (Step 313) . 

If it is judged that the windows are different (Step Sll, 
xv dif f erent") , the time necessary to redraw the Window is added 
to which O3J[0] oelcng (90,000 * (SIZE ( DSn . WDS . OBJ [ 0 ] . 
window id) ) // 256, 00C, 000) to the decode_durat ten (Step 315), the 
WAIT function is called using 0BJ[1] as an argument, and add a 
return value wait__duration to the decode_dufat ion (Step S16) , and 
the time necessary to redraw the Window to which OBJ [ 1 } belong 
: 90, COO * (SIZE (DSn . WDS . OBJ [0 j - window_id) ) / / 256, DOC, COG) to the 
de co de__du ration (Step S17) . 

The decode_duration is calculated by the above algorithm. 
A specific manner in which the-PTS of the OCS is set is explained 
below . 

FIG. 21A illustrates a case in which one ODS is included in 
one Window. FIGS.2IB and 21C are timing charts showing values 
in an order of time that are referred to in FIG . 18. A bottom line 
"ODS Decode" and a middle line "Graphics Plane Access" in each 
chart indicate two operations that are performed simultaneously 
when reproducing. The above algorithm is described assuming that 
these two operations are performed in paraile^.. 
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The Graphics Flans Acces s includes a clearing period (1) 
arc a write period (3) . The clearing period (1) indicates eizher 
a ~ ime period necessary :c clear an entire Graph! cs Plane 
' 9 : , C C 0 x (size of Graphics Plane //256,000,C3CI ; , cr a time period 
5 necessary to clear all Windows on the Graphics Plane 
:Z : . 92- , 000* (size cf window [i] / /2 5 6 , 0CG , 003 ) } . 

The write -period (3 5 indicates a time pence necessary zc 
render an entire Window ( SO , 000* ( size of Window [i] 
//256, 000, 000) ) . 

10 Further, a decode period (2) indicates a zime period between 

the DT3 and the PT3 of the ODS . 

Lengths of the clearing period (1), the decode period (2), 
and the write period ( 3 ) nay vary depending on a range 10 be cleared, 
a size of ODS to be decoded, and a size of the Graphics Object 
15 no be written zo the Graphics Plane. For convenience, a starting 
point cf zhe decode period (2) in the drawing is the same as a 
starting point of zhe clearing period (1). 

FIG.21B illustrates a case in which the decode period (2) 
is long, and the decode_curati on equals zo a zotai cf the decode 
20 period (2) and the wrize period (3). 

FTG.21C illustrates a case in which the clearing period (1) 
is long, and the decode__durat ion equals to a zctai of the clearing 
period (1) and the write period (3) . 

FIGs . 22A to 22C iiluszrate a case in which two ODS is included 
25 in one Window. The decode period (2) in both FIGs.22B and 22C 
indicates a total time period necessary for decoding two Graphics. 
Likewise, the write period (3) indicates a total time period 
necessary for writing two Graphics to the Graphics Plane. 

Even though the number of ODS is two, iz is possible to 
30 calculate the decode duration in the same manner as in the case 
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cf FIG. 21. When the decode perked {3} for decoding the two ODS 
is long, the decode_durat ion equals zc a cotal of the decode period 
■2) and the 'write period (3) as shown in FIG. 223. 

v?her. che clearing period (1) is long, the deccde__duration 
equals tc a total of the clearing period (1) and the write period 
(3). 

FIG . 2 3A describes a case in which each of two windows includes 
an ODS. As in the previous cases, when the clearing period {1) 
is longer chan the decode period (3) for decoding che two ODS, 
che decode___durat ion equals co a total of the clearing perioc (1) 
and the decode period (2) . However, when the clearing period (1) 
is shorter than the decode period (3), in is possible to write 
to a first Window before the decode period (2 ) ends . Accordingly, 
Lhe decooe_duraticn does net equal to either cf a total of the 
clearing period (1) and the wrice period (3), or a total of the 
decode period !2) and the write period (3) . 

When a time period necessary for decoding a first ODS is 
a write period (31) and a time period necessary for decoding a 
second ODS is a write period (32) , F1G.23B illustrates a case in 
which che decode period (2) is longer than a total of the clearing 
period (1) and che write period (31). In this case, the 
decode_duration equals to a total of the decode period {2} and 
the write period (32).. 

FIG.23C illustrates a case in which a total of the clearing 
period (1) and the wrice period (31) is longer than the decode 
period (2) . In this case, the decode_duration equals to a total 
of the clearing period (1), the write period (31), and "the write 
period (32) . 

The size of the Graphics Plane is known from a model of the 
reproduction apparatus in advance. Also, the size of the Window, 
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and the size and number cf the CDS are also known at the authoring. 
Accordingly, it is possible to find tc which combination of time 
periods the decoce_duration equals: the clearing period (1) and 
the write period (3) , the decoae period (2) and the write period 
(3 ) , the decode period (2 ) and me write period (32) , or the clearing 
period (1), the write period '3) and the write oeriod ;32) . 

By seating the PTS of the CDS based on the calculation of 
the decode__dura~ion explained above/ it is ocssible tc 
synchronously display the graphics with the piciure data at a high 
accuracy . Such a sync display at a high accuracy becomes possible 
by defining ihe Window and limiting an area to re-draw tc the Window . 
Thus, introducing a concept: of Window into an authoring environment 
has a great significance. 

The following is an explanation about settings of the DTS 
and PTS of zhe WDS- in the 13 r. . The DTS of the WDS may be set so 
.as to satisfy an equation oeiow. 
DTS (DSn [WDS: ) ^DTS (DSn [PCS; ) 

On the other hand, the OTS of the WDS in the DSn indicates 
a deadline to start writing to the Graphics Plane. Because it 
is sufficient to write to the window on the Graphics Plane, the 
lime to start writing to the Graphics Plane is determined by 
subtracting a time length indicated by the PTS of the PCS from 
a time period necessary for writing the WDS . When a total size 
of the WDS is £SI ZE ( WDS .-WIN [ i ] ) , the time necessary for clearing 
and re-drawing is "£SIZE (WDS . WIN [i] ) //256,COG,000". When 
expressing at a time accuracy of 90.000 KHz, the time is 
"90, 000*£SIZE (WDS . WIN [ i 1 ) / /2'5 6 , 00C , 000" . 

Accordingly, it is possible to calculate the PTS of the WDS 
by the following equation. 
PTS (DSn : WDS] ) = 
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PTS (DSn [PCS] j - 30000 ^SSIZE (WDS .WIN [il ) 7/256,000,000 

The PTS indicated in the WDS is the deadline, and it ispossible 
to start writing to the Graphics Plane earlier than the PTS . In 
other words, as shown in FIG . 23, once decoding the ODS tcbe rendered 
in one of the windows, writing of the Graphics Object obtained 
by the decoding may start at this point. 

As described above, it is possible to assign the Window to 
any point of time on the reproduction timeline of the AVClip using 
the DTS and PTS added to the WDS . 

Explanations about an example of settings of the GTS and 
PTS in a Display Set based on the settings are give below, referring 
to specific example illustrated in FIGs. 24-25. The example is 
about a case in which subtitles are displayed by writing tc the 
Graphics Plane four times, and an update is performed for displaying 
each of two subtitles "what is blu-ray." and "blu-ray is 
everywhere." FIG. 24 illustrates shifts in time of the update in 
the example.. Until a point tl, NN what" is 'displayed, and "what 
is" is displayed after the tl till a t2, and then "what is blu-ray. " 
is displayed at a t3 . After a whole sentence of a first subtitle 
has appeared, a second subtitle "blu-ray is everywhere." is 
displayed at a t 4 . 

FIG . 2 5A illustrates four Display Sets that are described 
so as to perform the above explained update. A DS1 includes a 
PCS1.2 for controlling an update at the tl, a PDS1 for coloring, 
an ODS1 corresponding tc the subtitle "what is blu-ray.", and an 
END as an ending code of the DS1 > 

A DS2 includes a PCS1.2 for controlling an update at the 
u2, and an END. A DS 3 includes a PCS1.3 for controlling an update 
at a t3 and an END. A DS 4 includes a PCS2 for controlling an 
update at the t2, a PDS2 for color conversion, an ODS2 corresponding 
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eo the subtitle u blu-ray is everywhere . " , and an END. 

Referring to a liming chart in FIG .253, settings of DTS and 
PTS fcr each functional segment in "he four Display Sees are 
explained . 

5 The reproduction timeline in ehe timing chart is ehe same 

as the timeline in FIG. 24. Ir_ ehe timing chart of FIG . 2 5A, 
PTS (PCS! . I) , PTS(?CSI.2), ?TS{?CS1.3} # and PTS(?CS2) are 
respectively see at a display point tl for displaying "what", a 
display poin- t2 for displaying u wha: is", a display peine t 3 for 
10 displaying "what is blu-ray . " , and a display point 1 4 f or di sola vine 
"blue-ray is everywhere . " . Each PTS are set as above, because 
it is necessary that ehe control such as cropping described in 
each PCS is performed at the display point of each subtitle. 

PTS(COSl) and PTS(ODS2) are set so as zc indicate points 
15 that are calculated by subtracting decode__durat ion from the points 
indicated by PTS(?CS1.1) and ?TS(PCS2) f respectively, because 
PTS (PCS) is required to be set so as to satisfy a formula below. 
PTS (DSn [PCS] ) >DT3 (DSn [PCS] ) -decodeduration ( DSn) 

In EIC.25B, PTS (ODS2) is sez so as to indicate a point z5 
20 that comes before the point Z4 , and PTS (ODS1 ) is set so as to indicate 
a point tO that comes before the point tl. 

DTS(ODSl) and DTS(ODS2) are set so as to indicate points 
that are calculated by subtracting decode__dur ation from the points 
indicated by PTSfODSL)- and ?TS(ODS2) 7 respectively, because 
• 25 DTS (ODS) is required to be set so as to satisfy an equation below. 
PTS (DS ;ODSj : ) —DTS ( DSn [ODS j ] )4-^0, 000x ( SIZE ( DSn [ODS j j ) //Rd) 

In FIG.25B, PTS(ODS2) is set so as to indicate the point 
zd that comes before the point tO, and PTS(ODSl) is see so as to 
indicate a point that comes before the point tO . A relation 
30 indicated by DTS (ODS2 ) =PTS {ODSI ) is satisfied here. 
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3y seating a PTS of an CDS immediately after a PTS of a 
preceding ODS to be displayed earlier, the reproduction apparatus 
performs an operation in which the ODS is read out to the memory 
so as to overwrite the preceding ODS , and thus it is possible that 
the reproduction process is performed by a small size of memory. 
3y realizing such a reproduction process, choices for a memory 
size for a reproduction apparatus become wider. 

The DTS of PCS1.1 is set so as to be DTS (?CSI . 1) =DTS (ODSI) , 
because the value for the DTS of PCS1.1 may be any point before 
the point indicated by DTS(ODSl) . 

The PTS of ODS1, .the DTS of ODS2, and the PTS of the PCS1.2, 
?CS1 . 3, and PCS 2 are set at the point tO, so as to satisfy a relation 
indicated by an equation below. 

PTS (OD31 ) -DTS (ODS2 ; -PTS ( PCS1 . 2 ) = PTS (PCS1 . 3 ) =?TS { PCS2 ) 

This is because the value for the DTS of PCS1.2 and PCS1..3 
may be any points before the point indicated oy PTS(?CS1.3) , and 
the DTS of PCS2 may be any point before trie point indicated by 
DTS ( PCS2 ) . 

As explained above, it is possioie to perform update of a 
succeeding PCS as soon as the updating of a previous PCS is completed, 
by reading out more than one PCS at the same time. 

It is sufficient that the DTS and PTS of PCS and the DTS 
and PTS of ODS satisfy the relations indicated by the formulae 
above. Accordingly, it becomes possible that the values are set 
to be DTS (ODS2) -PTS (0DS1) or PTS (ODS1) -DTS (0DS2) - 

PTS iPCSl . 2) -PTS (PCS! . 3)-DTS (PCS2) . By such settings for time 
stamps, it is possible to adjust time length of a period in which 
, C3 d in decoding increases or more buffers are needed. Such 
adjustment expands possibility of the controls during the 
reproduction, and advantageous for those who perform authoring 
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or manufacture reproducing apparatuses. 

Da-La structures cf the Display Sets (PCS, WDS, PDS, ODS) 
explained above is an instance cf the class structure described 
in a proaramming language. Producers that perform authoring may 
obtain the data structures cn the BD-ROK by describing the class 
structure according to the syntax provided in' the Blu-ray Disc 
Prerecording Format . 

Next, a practical example of a reproduction apparatus 
according to the present invention is explained below. FIG. 26 
illustrates an internal structure of the reproduction apparatus 
according to the present invention. The reproduction apparatus 
according to the present invention is industrially produced based 
on the internal structure shown in the drawing . The reproduction 
apoaratus according to the present invention is mainly structured 
by three parts: a system LSI, a drive device, and a microcomputer 
•system., and it is possible to industrially produce the reproduction 
apparatus by mounting the three parts to a cabinet and a substrate 
of the apparatus . The system LSI is an integrated circuit in which 
various processing units for carrying out a function of the 
reproduction apparatus are integrated. The reproduction 
apparatus manufactured in the above manner comprises a BD drive 
1, a Read 3uffer 2, a PID filter 3, Transport Buffers 4a-4c, a 
peripheral circuit 4d, a Video Decoder 5, a Video Plane 6, an Audio 
Decoder 7, a Graphics Plane 8 , a GLUT unit 9, an adder 10, a Graphics 
Decoder 12, a Coded Data Buffer 13, a peripheral circuit 13a, a 
Stream Graphics Processor 14, an Object Buffer 15, a Composition 
Buffer 16, ana a Graphical Controller 17. 

The BD drive 1 performs load/read/eject of the BD-ROM, and 
accesses to the BD-ROM. 

The Read Buffer 2 is a FIFO memory for storing the TS packets 
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read frorr. the 3D- ROM in afirst-in first-cut order . 

The PID filter 3 filters more than one TS packet outputted 
from -he Read Buffer 2. The filtering by the PID filter 3 is to 
write the only TS packets having a desired PID to the Transport 
Buffers 4a-4c. Buffering is not necessary for the filtering by 
the PID filter 3, and accordingly, the TS packets inputted tc the 
PID filter 3 are written to the Transport Buffers 4a-4c without 
delay . 

The Txans£ori34^fers 4a -4c are for storing tne TS packets 
outputted frorr. the PID filter 3 in a first-in first-out order. 
A speed at which the TS packets frorr. the Transport 3uffers 4a-4c 
are outputted is a speec Rx . 

The peripheral circuit 4d is a wired logic for converting 
the TS packets read from the Transport Buffers 4a-4c into functional 
segments . The functional segments obtained by the conversion are 
stored in the Coded Data Buffer 13. 

The Video Decoder 5 decodes the more than one TS packets 
outputted from the PIC filter 3 into a decompressed picture and 
writes tc the Video Plane 6. 

The Video Plane 6 is a plane memory for a moving picture. 

The Audio Decoder 7 decodes the TS packets outputted from 
the PID filter 3 and outputs decompressed audio . data. 

The Graphics Plane 8 is a plane memory having an area for 
one screen, and is able to store decompressed graphics for one 
screen . 

The CLUT unit 9 converts an index color of the decompressed 
Graphics stored in the Graphics Plane 8 based on the values for 
Y, Cr, and Cb indicated by the PDS . 

The adder 10 multiplies the decompressed Graphics to which 
the color conversion has been performed by the CLUT unit 9 by the 
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7 value (Transparency) indicated by the ?DS, adds the decomposed 
picture data storec in the video Plane per pixel, then obtains 
and outputs the composed image. 

The Graphics Decooer 12 decodes the Graphics Stream to obtain 
the decomposed graphics, and writes the decomposed graphics as 
the Graphics Object :c the Graphics Plane 8. By decoding the 
Graphics Stream, the subtitles and menus appear on zhe screen. 
The Graphics Decoder 12 includes the Coded Data Buffer 13, the 
peripheral circuit 13a, the Stream Graphics Processor 14, the 
Object Buffer 15, the Composition Buffer 16, and zhe Graphical 
Controller 17. 

The Coded Data Buffer 13 is a buffer in which zhe functional 
segment is stored along with the DTS and ?TS . The functional 
segment is obtained by removing a TS packet header and a PES packet 
header from each TS packet in the Transport Stream stored in -the 
Transport Buffer 4 a- 4 c and by arranging the payloads sequentially . 
The ?TS and DTS out of the removed TS packet header and PES packet 
header are stored after making correspondence between the PES 
packets . 

The peripheral circuit 13a is a wired logic that realizes 
a transfer between the Coded Data Buffer 13 and the Stream Graphics 
Processor 14, and a transfer between the Coded Data Buffer 13 and 
the Composition Buffer 16. In the transfer operation, when a 
current time is a time -indicated by the DTS of the ODS , the ODS 
is transferred from the Coded Data Buffer 13 to the Stream Graphics 
Processor 14. When the current time is a time indicated by the 
DTS of the PCS and PDS, the PCS and PDS are transferred to the 
Composition Buffer 16. 

The Stream Graphics Processor 14 decodes the ODS, and writes 
the decompressed graphics of the index color obtained by decoding 
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as the Graphics Object tc the Oojecc Buffer 15. The decoding by 
che Scream Graphics Processor 14 starts at the time of the CTS 
corresponding co the ODS, and ends by the decode end time indicated 
by the PTS corresponding to the CDS. The decoding rate Rc of the 
Graphics Object is an oucpuc rate of the Scream Graphics Processor 
14 . 

The Object Buffer 15 is a buffer corresponding tc a pixel 
buffer in che STSI EN 300 7 43 scandard, and the Graphics Objecc 
obtained by che decode chat the Stream Graphics Processor 14 
performs is disposed. The Objecc Buffer 15 needs tc be sec to 
twice or four times as large as the Graphics Plane 8, because in 
case che Scrolling effect is performed, the Object Buffer 15 needs 
to store tne Graphics Object rhac is twice or four times as large 
as the Graphics Plane. 

The Composition Buffer 16 is a memory in which che PCS and 
PDS are disposed. 

The Graphical Controller 17 decodes the PCS disposed in the 
Compos icior. Buffer 16, and performs a control based on che PCS. 
A timing for performing the control is based on che PTS accached 
tc the PCS. 

N*exc, recommended values for the transfer rate and ouffer 
size for structuring the PID filter 3, Transport Buffer 4a-4c, 
Graphics Plane 8, CULT unit 9, Coded Data Buffer 13, and Graphical 
Controller 17 are explained . FIG. 2 7 illustrates sizes of the write 
rates Rx, Rc, andRd, Graphics Plane 8, Coded Data Buffer 13, Ooject 
Buffer 15, and Composition Buffer 16. 

The transfer rate Rc between the Object Buffer 15 and tne 
Graphics Plane 8 is the highest transfer rate in che reproduction 
apparatus of the present embodiment, and calculated as 256Mbps 
( = 500 Kbytes * 2S.97 * 2) from the window size and the frame race. 
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unlike the Rc, the transfer rate Rd (Pixel Decoding Rate) 
between the Stream Graphics Processor 14 and Object Buffer 15 does 
no: need to be updated every video frame cycle, and 1/2 or 1/4 
of the Rc is sufficient for z'r.e Rd. Accordingly, the Rd is either 
5 128 Mops or 64 Mbps . 

The Transport Buffer Leak Rate Rx between the Transport 
Buffer 4a-4c and Coded Data Buffer 13 is a transfer rate of the 
CDS in a compressed state. Accordingly, the transfer rate Rd 
multiplied by the compression rate is sufficient for the Transport 
10 Buffer leak rate Rx . Given the compression rate of the ODS is 
25%, 16 Mbps {=64 Mbps * 25%) is sufficient. 

The transfer rates and buffer sizes shown in the drawing 
are the minimum standard, and it is also possible to set at higher 
rates and larger sizes. 
15 In the above structured reproduction apparatus, each 

elements perform a decoding operation in a pipeline structure. 

FIG. 28 is a timing chart illustrating a pipeline processing 

by the reproduction apparatus . A 5th row in the drawing is a Display 

>r 

Set in the BB-ROM, a 4th row shows read periods from the PCS, WDS , 
r- — . — 

20 ?DS , and ODS to the Codec Data Buffer 13. A 3rd row shows decode 
periods of each ODS by the Stream Graphics Processor 14. A 1st 
row shows operations that the Graphical Controller 17 performs. 

The DTS (decode starting time) attached to the ODS1 and ODS2 
indicate t31 and 1 52 in the drawing, respectively. Because the 

25 decode starting time is set by DTS, each ODS is required to be 

read out to the Co ded Data Buff er 13. Accordingly, the reading 

of the 0DS1 is completed before a decode period dpi in which the 

ODS1 is decoded to the Coded Data Buf fer 13 . Also, the reading 

of the ODS2 is completed before a decode period dp2 in which the 
«— ^ ^ 

30 ODS2 is decoded to the Coded Data Buffer 13. _ 
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On the other hand, the PTS ; decode ending time) attached 
to the 0DS1 ar.dODS2 indicate t3 2 and t 3 3 in the drawing , respectively. 
Decoding of the 0DS1 by the Stream Graphics Processor 14 is completed 
by che t32 , and decoding of the ODS2 is completed by a time indicated 
by the t33 . As explained above, the Scream Graphics Processor 
14 reads the ODS to the Coded Data Buffer 13 by the time the CDS 
of the ODS indicates, and decodes the ODS read to the Coded Data 
Buffer 13 by che cime the PTS of the ODS indicates, and write the 
decoded ODS to che Objecc Buffer 15, 

A perioc cdl at ihe 1st row in the drawing indicates a period 
necessary for the Graphics Controller 17 to clear the Graphics. 
Plane. Also, a period tdl indicates a period necessary to write 
the Graphics Object obtained on ~he Object Buffer cc the Graphics 
Plane 8 . The PTS of the KDS indicates che deadline to stare writing , 
and the PTS of the PCS indicates ending of the wriie and a timing 
for display. At che time indicated by the PTS of che PCS, the 
decompressed graphics to compose an interactive screen is obtained 
on the Graphics Plane 8 . 

After the CLUT unit 9 performs the color conversion of the 
decompressed graphics and the adder 10 performs composition of 
the decomposed graphics and a decomposed picture stored in the 
Video Plane 6, a composite image is obtained. 

In the Graphics Decoder 12, the Stream Graphics Processor 
14 performs decoding continuously while the Graphics Controller 
17 performs clearing of the Graphics Plane 8 . By the above pipeline 
processing, it is possible to perform a prompt display of the 
graphics. 

In FIG. 28, a case in which the clearing of the Graphics Plane 
ends before completing the decoding of the ODS is explained . FIG . 2 9 
illustrates a timing chart in a pipeline processing of a case in 
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which the decoding of the COS ends before the clear ing of the Graphics 
Plane is completed. In this case, it is not possible tc write 
to the Graphics Plane at a time of completion of the decoding of 
the CDS. When the clearing of the Graphics Plane is completed, 
tc becomes possible to write the graphics obtained by the decode 
tc the Graphics Plane. 

Next, how the controlling unit 20 and the Graphics Decoder 
12 are implemented is explained below. The controlling unit 2C 
is implemented by writing a program performing an operation shown 
in FIG . 3 C , and having a general CPU execute the program. The 
operation performed by the controlling unit 20 is explained by 
referring, to FIG. 30 . 

TIG . 3 0 is a flowchart showing a process of a loading operation 
of the functional segment . In the flowchart , SegmentK is a variable 
indicating each of Segments (PCS, WDS, PDS, and ODS) that is read 
out in reproducing the AVClip. An ignore flag is a flag to determine 
if the SegmentK is ignored or loaded. The flowchart has a loop 
structure, in which first the ignore flag is initialized to C and 
then Steps S21-S24 and Steps S27-S31 are repeated for each SegmentK 
(Step S25 and Step 326). 

Step S21 is for judging if the SegmentK is the PCS, and if 
the SegmentK is the PCS, judgments in Step S27 and Step S28 are 
performed . 

Step S22 is for judging if the ignore flag is 0 . If the 
ignore flag is C, the operation moves to Step S23, and if the ignore 
flag is 1, the operation moves to Step S24. If the ignore flag 
is C (Yes in Step S22) , the SegmentK is loaded to the Coded Data 
Buffer 13 in Step S23. 

If the ignore flag is 1 (No in Step S22) , the SegmentK is 
ignored in Step S24. 3y this, the rest of all functional segments 
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that helono to the DS are ignored because Step S22 is No (Step 
S24) . 

As explaineo aoove , whezher the SegmeniK :s ignored or loaded 
is determined by the ignore flag. Steps S27-S31, S34, and S35 
are steps for setting the ignore flag. 

In Steo 527, it is judged if segmet_type of the SegmentK 
is the Acquisition Point . if the SegmentK is the Acquisition Point, 
the operation moves to Step S28, and if the SegmentK is either 
the Enoch St am cr Normal Case, then the operation moves to Step 
S31 . 

In Step S28, it is judged if a preceding DS exists in any 
of the buffers in the Graphics Decoder 12 (the coded data buffer 
13, stream graphics processor 14 , obj ect buff er 15, and composition 
buffer 16) . The judgment in Step S26 is made when the judgment 
in Step S27 is Yes. A case in which a preceding DS does not exist 
in the Graphics Decoder 12 indicates a case in which the skip 
operation is performed. In this case, the" display starts from 
the DS that is the Acquisition Point, and therefore the operation 
moves to Step S30 :Nc in Step S28) . In Step S30, the ignore flag 
is set to C and the operation moves tc Step S22. 

A case in which a preceding DS exists in the Graphics Decoder 
12 indicates a case in which normal reproduction is performed. 
In this case, the operation moves to Step S2S (Yes in Step S28) . 
In Step S29, the ignore flag is set to 1 and the operation moves 
to Step S22 . 

In Step S31, it is judged if segment__type of the PCS is the 
Normal Case. If the PCS is the Normal Case, the operation moves 
to Step S34, and if the PCS is the Epoch Start, then the ignore 
flag is set to 0 in Step S30. 

In Step S34, like in Step S28, it is judged if a preceding 
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DS exists in any of the buffers in the Graphics Decoder 12. If 
the preceding DS exists, the ignore flag is set tc 0 (Step S30) . 
If the preceding DS does not exist, it is not possible tc obtain 
sufficient functional segment s to compose an interactive screen 
and the ignore flag is set to 1 (Step S35) . 

3y setting the ignore flag in the above manner, the functional 
segments that constitute the Normal Case are ignored when the 
preceding DS does not exit in the Graphics Decoder 12. 

Taking an example of a case in which the DS is multiplexed 
as shown in FIG. 31, a manner how the reading of the DS is performed 
is explained. In the example of FIG . 31 , three DS are multiplexed 
with a moving picture. The segment__t ype of a DSi is Epoch Start, 
the segment__type of a DS10 is Acquisition Point, and the 
segment___type of a- D320 is Normal Case. 

Given that, in an AVClip in which the three DS and the moving 
picture are multiplexed, a skip operation to a picture data otIO 
as shown by an arrow amlis performed, the DS10 is the closest to 
a skipping target, and therefore the DSIC is the DS described in 
the flowchart in FIG .30. Although the segment__type is judged tc 
be the Acquisition Point in Step S27, the ignore flag is set tc 
C because no preceding DS exists in the Coded Data Buffer 13, and 
the DSIC is loaded to the Coded Data Buffer 13 of the reproduction 
apparatus as shown by an arrow mdi in FIG. 32. On the other hand, 
in a case in which the skipping target is after the DSI 3 (an arrow 
au\2 in FIG . 31 ) ,- the DS20 is to be ignored because the DS20 is Normal 
Case Display Set and DS2C because a preceding DS does not exist 
in the Coded Data Buffer 13 (an arrow md2 in FIG. 32) . 

FIG. 33 illustrates loading of the DSi, DS10, and DS20 in 
a normal reproduction. The DSI whose segment_type of the PCS is 
the Epoch Start is loaded to the Coded Data Buffer 13 as it is 
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;Step S23) . However, because "he ignore flag cf the DS10 "whose 
segment__type of "he PCS is the Acquisition Point is-set to 1 (Step 
S2 9 ) , the functional segments that constitute zhe CS10 are ignored 
and not loaded to the Coded Data Buffer 13 (an arrow rd2 in FIG. 34 , 
and Step S24 ) . Further, the DS20 is ioadeo to the Codec Data Buffer 
13, because the segment_type of the PCS of the DS2C is the Normal 
Case (an arrow rd3 in FIG .34) . 

Next, operations by the Graphical Controller 17 are explained . 
FIGs .35-37 illustrate a flowchart showing the operations perf ormed 
by the Graphical Controller 17. 

Steps S41-S44 are steps for a main routine of the flowchart 
and waits for any cf events prescribed in Steps S41-S44 occurs. 

Step S41 is to judge if a current reproducing time is a time 
indicated by the DCS of the PCS , and if the judging is Yes, then 
an operation in Steps S4 5-353 is performed. 

Step S45 is to judge if the compcsition_state of the DCS 
is the epoch start , and if judged to be the epcch_s tart , the Graphics 
Plane 8 is all cleared in Step S46. If judged to be other than 
~- ne epoch start, the Window indicated by the 
window horizontal position, window_vert ical__position , 

window width, and wi ndow__he i gh t of the WDS is cleared. 

Step S48 is a step performed after the clearing performed 
in Step S46 or in Step S47, and to judge if the time indicated 
by the PTS of any ODSx has passed. The decoding of any ODSx couId 
be already completed by the time the clearing ends, because the 
clearing cf an entire Graphics Plane 8 takes time. Therefore, 
in -Steps S4 8, it is judged if the decoding of any ODSx is already 
completed by the time the clearing ends. If the judging is No, 
the operation returns to the main routine. If the time indicated 
cy the PTS of any ODSx has already passed, an operation in Steps 
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34 9-S51 is performed . In Seep 34 9, it is judged if oh i ec; _croc_f lag 
is 0, and if the flag indicates C , then the Graphics Object is 
set tc "no display" (Step S50) . 

If the flag is not Z ir. S~ep S49, then an object cropped 
5 based on ob j ect_cro?ping_horizontal_positicn, object 
_cropping_vertical__posicior. , cropping_width, and cropping 
_height is written to the window in the Graphics Plane 8 at the 
position indicated by ob j ect_cr cpping_hor i zcntal_pos icion and 
cbj ect_cropping_vertical_po5it con (Step 351) . By the above 

10 operation, one or more Graphics Objects are rendered in the Window. 

In Step 52, it is judged if the time corresponding tc a ?TS 
of another ODSy has passed. When writing the ODSx. to the Graphics 
Plane 8, if the decoding of the ODSy has already been completed, 
chen the ODSy becomes ODSx (Step S53), and the operation moves 

15 cc Step S49. By this, the operation from Steps S49-S51 is also 
performed to another ODS . 

N-ext, by referring to FIG .36, Step S42 and Steps 554-S59 
are explained below. 

In Step 42, it is judged if the current reproducing point 

20 is at the ?TS of the WD3 . If the judging is that the current 
reproducing point is at the PTS of the WDS, then it is judged if 
the number of the Window is one or not in Step 354. If the judging 
is two, the operation returns to the main routine. If che judging 
is one, a loop processing of Steps S55-S59 is performed. In the 

25 loop processing, operations in Steps S55-S59 are performed to each 
of the two Graphics Object displayed in the Window . In Step S57, 
it is judged if ob j ect_crop_f lag indicates 0. If if indicates 
0, then the Graphics is not displayed (Step S58). 

If it doesn't indicate 0, then a cropped object based on 

30 ob j ect_cropping_horizontal_position, ob j ect__cropping 
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_vertical_posit ion, cropping__width , and cropping_height is 
written to the Window in the Graphics Plane 8 at the position 
indicated bv ob j ecu_croppin g__hori z on t a Imposition and 
cbj ect_cropping_vertical_position (Step S5S) . By repeating the 
5 above operations, more than one Graphics Object is rendered in 
the Window. 

In Steo 544, it is judged if the current reproducing point 
is at tne PTS of the ?DS . If the judging is thai, "he current 
reproducing point is at the ?TS of the PDS, then it is judgec if 
10 pallet__update_f lag is one or not in Step S6C. If the judging is 
one, the PDS indicated by pallet__id is set in the CLUT ur.ii (Step 
S61) . If the judging is 0, then Step S61 is skipped. 

After that, the CLUT unit performs the color conversion of 
the Graphics Object on the Graphics Plane 8 to be combined with 
15 the moving picture (Step S62) . 

Next, 'by referring to FIG. 3*7, Step S43 and Steps S64-S66 
are explained below. 

In Steo 43, it is judged if the current reproducing point 
is at the PTS of the ODS . If the judging is that the current 
20 reproducing point is at the PTS cf tne COS, then it is judged if 
the number of the Window is two or not in Step S63. If the judging 
is one, the operation returns tc the main routine. If the judging 
is two, operations in Steps S64-S66 are performed. In Step SG4, 
it is judgec if ob j ect_crop__f lag indicates 0. If it indicates 
25 0, then the Graphics is not displayed (Step S65) . 

If it doesn't indicate 0 then a cropped object based on 
ob j ect__crcpping_horizontal__position, obj ect__cropping 

vertical position, cropping_width , and cropping___h eight is 
written to the Window in the Graphics Plane 8 at the position 
30 indicated by obj ect__cropping_hori zontaI___posit ion and 
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oc; ect_cropping_verticai_jpos ition (Step S66) . By repealing the 
above operations, the Graphics Object is rendered in each Window. 

The above explanations are about the DTS and PTS of the PCS, 
and -he CTS and PTS of the CDS that belong tc DSn. The DTS and 
PTS of the PDS, and the DTS and PTS of the END are not explained. 
Firs- , the DTS and PTS of the ?D that belongs to the DSn are explained . 

As for the PDS that belongs to the DSn, it is sufficient 
if the PDS is available in zhe GLUT unit 9 by the PCS is loaded 
tc the Composition Buffer 16 ( DTS ( DSn [ PCS ] ) ) after decoding start 
point of a first ODS ( DTS ( DSn [ODS1 ] ) ) . Accordingly, a value of 
PTS of each PDS [ PDS 1- PDS last ) in the DSn is required :o be set 
so as tc satisfy the following relations. 
DTS (DSn [PCS] ) <?TS (DSn [?DS1." ! 

FTS ( DSn I PDS j ] ) <PTS ( DSn [PDSj + 1 ] )<?TS ( DSn [ PDSlast ] ) 
PTS (DSn [PDSlast] } <DT3 (DSniODSl] ) 

Note chat the DTS of the PDS is not referred ic during the 
reproducing, the DTS of the CDS is set to the same value as the 
PTS of -he PDS in order to satisfy the MPEG2 standard. 

Following is an explanation about roles of zhe DTS and PTS 
in zhe pipeline processing of the reproduction apparatus when the 
DTS and PDS are set so as to satisfy the above relations. FIG. 38 
illustrates the pipeline of the reproduction apparatus based on 
the PTS of the PDS. FIG . 38 is based on FIG. 26. A first row in 
FIG. 38 indicates setting the ODS in the CLUT unit 9. Under the 
first row are the same as first to fifth rows in FIG . 26 . The setting 
of the PDSl-PDSlast to the CLUT unit 9 is performed after the 
transferring the PCS and WDS and before the decoding of the ODS1, 
and accordingly the setting of the PDSl-PDSlast to the CLUT unit 
9 is set before a point indicated by the DTS of the ODS1 as shown 
by arrows up 2 and up3 . 
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As described above, the serine of the PDS is performed in 
prior :c the decoding of the ODS . 

Next, a setting of the PTS cf END of Display Set segment 
in the DSn is explained. The END that belongs tc the DSn indicates 

5 the end of "he DSn, and accordingly it is necessary that one PIS 
of ohe END indicates the decode ending time of the 0DS2 . The cecode 
ending oime is indicated by ohe PTS ( PTS (DSn [ODS last] ) i cf the 
ODS2 .(ODSlasi) , and therefore the PTS of the EKD is required to 
be set ao a value that satisfies an equation below. 

10 DTS { DSn [EKD] ) = PTS ( DSn [ODSlaso] } 

In terms of a relation between ohe DSn and the PCS that belongs 
to the DSn-1 , ohe PCS in the DSn is loaded to the Composition. Buffer 
16 oefore a loading time of the first ODS (ODS1) , and therefore 
the PTS of one EKD should be after a loading time of the PCS in 

15 ohe DSn and before a loading time of ohe PCS that belongs oc the 
DSn+l. Accordingly, the PTS of the END is required to satisfy 
a relation below. 

DTS ( DSn [ PCS ] ) <PTS (DSn [END] )<DTS (DSn+I [PCS] ) 

On the other hand, the loading time of the first ODS (ODSI) 
20 is before a loading time of a last PDS (PDSlast) , and therefore 
the PTS of the END ( PTS ( DSn [ END ] ) ) should be after a loading line 
of tne PDS that belongs to the DSn (PTS ( DSn I PDS las o] ) ) . 
Accordingly, the PTS cf the END is required to satisfy a relation 
be lev;. 

25 PTS ( DSn [ PDSlast ] ) ^PTS (DSn [END; } 

Following is an explanation about significance of the PTS 
of the END in the pipeline processing of the reproduction apparatus . 
TIG. 39 is a diagram describes the significance of the END in the 
pipeline process of the reproduction apparatus. FIG . 39 is cased 
30 on FIG. 26, and each row in FIG . 3 9 is substantially the same as 
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FIG. 2 6 other ihar. that a first row in FIG. 3 5 indicates the content 
of the Composition Buffer 16. Further, in FIG . 39, 2 Display Sets, 
DSn and DSn+L are illustrated. The ODSlast in uhe DSn is the last 
ODSn of A-ODSs, and accordingly, the point indicated by the PIS 
of the END is cefore the DTS of zhe PCS in the DSn+1. 

By the FTS of the. END, it is possible to find when the loading 
of the ODS in the DSn is completed during reproduction. 

Note that although the DTS of uhe END is not referred to 
during reproduction, uhe DTS of uhe END is set tc the same value 
as the PTS of uhe END in order to sauisfy the MPEG2 standard. 

As described in the above, a pari of the Graphics Plane is 
specified as the Window for displaying the Graphics according :o 
the present embodiment, and uheref ere uhe reproduction apparatus 
does nou have to render uhe Graphics for an entire Plane. The 
reproduction apparatus may render uhe Graphics for only a 
predetermined size of Window, such as 25% to 33% of the Graphics 
Plane. Because the rendering of uhe Graphics ouher than the 
Graphics in uhe Window is not necessary, uhe load for sofuware 
in the reproduction apparatus decreases. 

Even in a worst case in which the updating of the Graphics 
is performed such as 1/4 of uhe Graphics Plane, it is possible 
tc display uhe Graphics synchronously with the picture by the 
reproduction apparatus performing the wriue to the Graphics Plane 
at a predetermined uransfer rate such as 256 Mbps, and by setuing 
the size of the Window so as to ensure the sync display with the 
picture . 

Thus, it is possible to realize a high resoluuion subtitle 
display for various reproduction apparatuses, because the sync 
display is easily ensured. 
[ Second Embodiment ] 
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In the firsr embodiment , the size of the Window is set tc 
1/4 of an entire Graphics Plane and the writing rate Rc zo the 
Grachics Plane is set to 256 Mbps, so as tc update zhe Graphics 
for each video frame. Further, by setting the update rate to be 
1/2 or 1/4 of zhe video frame rate, it 'oecomes possible tc update 
a laraer size of the Graphics. However, when zhe update rate is 
1/2 or 1/4 of the video frame rate, it zakes 2 cr 4 frames to write 
zc zhe Graphics Plane. When one Graphics Plane is provided, a 
process of the writing of the Graphics during zhe 2 or 4 frames 
during which the Graphics is written becomes visible to a user. 
In such a case, a display ef f ecz such as switching from one Graphics 
zo a larger Graphics in a moment may noz be ef feczively realised. 
Therefore, in a second embodiment, two Graphics Planes are provided. 
FIG. 40 illustrates an internal structure of a reproduction 
apoaratus according zc zhe second embodiment. The reproduction 
aooaratus in FIG . 4 0 is new in comparison wizh the reproduction 
apparatus according to FIGs.2 4 and 25 in zhaz the reproduction 
apparatus in FIG. 40 has two Graphics Planes (a Graphics Plane 81 
and a Graphics Plane 82 in zhe drawing) , and the two Graphics Planes 
constitute a double buffer. Accordingly, it is possible to write 
zo one of the Graphics Planes while the reading is performed from 
the other of the Graphics Planes . Further, a Graphical Controller 
17 according to the second embodiment switches the Graphics Plane 
-hat is read out at a point indicated by the PTS of the PCS. 

FIG. 41 schematically illustrates an operation of reading 
ouz and writing to the Graphics Planes that constitute the double 
buffer. An upper row indicates contents of the Graphics Plane 
81, and a bottom row indicates contents of the Graphics Plane 82. 
The contents of the both Graphics Planes per frame are illustrated 
from a first frame tc a fifth frame [left to right) . A part of 
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the Graphics Planes 81 and 8 2 for each frame than are enclosed 
by a -hick line is a target of the reading our. In the drawing, 
a face -ark is contained in the Graphics Plane 81, and the face 
mark is ;o oe replaced by a sun mark that is in an Object 3uf fer 
15. A size of the sun mark is 4 Mbytes, which is a maximum size 
cf the Object Buffer 15. 

To write one sun mark tc the Graphics Plane 82 at zhe writing 
rate re the Graphics Plane [Rc=2 5 6 Mbps), it takes 4 frames till 
the writing is completed, and only 1/4 of the sun mark is written 
to the Graphics Plane 82 during the first frame, 2/4 curing the 
second frame, and 3/4 during the "hire frame . 3ecause the Graphics 
Plane Bl is the target tc be displayed in the screen, however, 
the process of writing the sun mark to the Graphics Plane is not 
visible tc the user. At the fifth frame, when the target cf display 
switches to the Graphics Plane 32, the contents cf the Graphics 
Plane 82 become visible tc the user. Thus, the switching from 
the face mark to the sun mark is completed. 

As described above, according to rhe second err.be diment , it 
is oossible to switch display in the screen to a nor her graphics 
at once even when a large size graphics is wrirren to rhe Graphics 
Plane for four frames, and therefore useful when displaying such 
as credir s , an ourline of a movie, or a warning, at once in an 
entire screen. 
[Third Embodiment] ' ■ 

A third embodiment relates to a manufacturing process of 
the BD-ROM. FIG. 42 is a flowchart illustrating the manufacturing 
process of the BD-ROM according to the third embodiment. 

The manufacturing of the BD-ROM includes a material 
manufacturing step S2 01 for producing material and recording movies 
and sound, an authoring step S202 for generating an application 
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format using an authoring apparatus, and a pressing step 5203 for 
manufacturing a master disc of the 3D- ROM and pressing tc finish 
the BD-ROM. 

Tne authoring step of the BD-ROM includes Steps S204-S209 
as fellows. 

In Step S204, the WDS is described so as to define the Window 
in which subtitles are displayed, and in Step S2C5, a period of 
time during which the window is defined to appear at the same position 
in tne same size, is set as one Epoch, and the FCS for each Epoch 
is described. 

After obtaining the OCS in the above manner, tne Graphics 
as material for subtitles is converted into the CDS, and the Display 
Set is obtained by combining -he ODS with the PCS, WDS , and ?DS 
in Steo 3206. Then, in Step S2C7, each functional segment in the 
Display Set is divided into the PES packets, and the Graphics Stream, 
is obtained by attaching the time stamp. 

Finally, in Step S208, the AVClip is generated by 
multiplexing the graphics stream with the video stream and audio 
stream that are generated separately. 

After obtaining the AVClip, the application format is 
completed by adjusting the AVClip into the BD-ROK format. 
[Other Matters] 

The above explanations do not illustrate ail embodiments 
according to the present invention. The present invention may 
also be realized by modified examples shown below. Inventions 
described in the Claims of the present application include the 
above embodiments as well as expansions or generalizations of the 
modified examples. Although the degree of expansion and 
generalization is based on characteristics of technological levels 
of the related art at the time of the application, the inventions 
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according :o the Claims of ~he present application reflect the 
means tc solve the technical problems in the conventional art, 
and therefore the scope of the invention does not exceed the 
technological scope that those skilled in the art would recognize 
5 as means to solve the technical problems in the conventional art. 
Thus, the inventions according to the Claims of the present 
application substantially correspond to the descriptions of the 
details of the invention. 

(1) The BD-ROM is used in the explanations of all of the 
10 above embodiments. However, characteristics of the present 

invention are in the Graphics Stream that is recorded in a media, 
and such characteristics do not depend on physical properties of 
the ED-ROM, Any recording medium that is capable of storing the 
Graphics Stream may realize the present invention. Examples of 

15 such recording medium include optical discs such as a DVD-ROM, 
a DVD-RAM, a DVD-RW, a DVD-R, a DVD+RW, a DVD+R, a CD-R, and a 
CD-RW, magnetic optical discs such as a PD and MO, semiconductor 
memory cards such as a compact flash card, a smart media, a memory 
stick, a multi-media card, and a PCM-C1A card, and magnetic discs 

20 such as a flexible disc, a SuperDisk, a Zip, and a Clik! , and 
removable hard disk drives such as an ORB, a Jaz, a SparQ, a SyJet, 
a EZFiey, and a micro drive, in addition to built-in hard disks. 

(2) The reproduction apparatus described in ail of the 
above embodiments decodes the AVClip recorded in the 3D-R0M and 

- 25 outputs the decoded AVClip to a TV. However, it is also possible 
to realize the present invention by the reproduction apparatus 
that includes only a BD-ROM drive, and the TV provided with other 
elements. In this case, the reproduction apparatus and the TV 
may be connected via IEEE1394 to create a home network. Moreover, 
30 although the reproduction apparatus in the embodiments is used 
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bv. connecting to the 7V, the reproduction apparatus may be an C LI 
in one TV anc reproduction apparatus. Further, the LSI {integrated 
circuit:) alone that forns an essential part of the processing in 
the reproduction apparatus of each embodiment may be pur into 
cractice. Such reproduction apparatus and the LSI are both 
described in the present specification, and therefore 
manufacturing a reproduction apparatus based on the internal 
structure of the reproduction apparatus according to the first 
embodiment is an implementation of the present invention, no matter 
what working example it may take. Moreover, transferring whether 
as a cift or profit, lending, and importing of the reproduction 
apparatus according to the present invention are also considered 
to be the implementation of the present invention. Offering such 
transfer and lend to common users by way of storefront display 
and distribution of brochure is also considered to ce the 
imoiementat ion of the present invention. 

(31 Information processing executed by a program snown 
in the flowcharts is realized using hardware resources, and 
accordingly, the program whose processing is shown in each 
flowchart is established alone a's an invention. Although all the 
above embodiments describe the program according to the present 
invention as built-in in the reproduction apparatus, the program 
according to the first embodiment alone may be implemented. 
Examples of implementation of the program alone include (i) 
producing the programs, (ii) transferring the programs as a gift 
orprofit, (iii) lending the programs, ( iv) importing the programs , 
(v) providing general public with the programs via an interactive 
electronic communications line, and'(vi) offering the transfer 
and ' lend to common users by way of storefront display and 
distribution of brochure. 
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(4) Time elements in the steps that are performed in a 
sequential order in each flowchart are essential characteristics 
of the pre sen" invention, and it is clear that the process shown 
in each f lowcha rts discloses a method of reproduction . Performing 
the processes illustrated by the flowcharts by carrying out the 
operation in each step sequentially to obtain the object of the 
present invention and realizes the effecrs is the implementation 
of the recording method according to the present invention. 

(5) It is desirable to add an extension header to each 
packets constituting the AVClip when recording in the BD-ROM * The 
extension header is 4 -byres data called TP_extra_header that 
includes arrival _r ime_stamp and c c p y __p er mission ^indicator . The 
TS packets having the T?__ext ra_header (hereinafter referred to 
as the TS packers, with EX) are grouped by every 32 packers and 
written to 3 sectors. A group including 32 TS packets with EX 
has 6144 bytes (=32*192), which is r he same size as a size of 3 
sectors 6144 bytes (=2048x3) . The group of 32 TS packets with 
EX stored in 3 sectors is called Aligned Unit-. 

When the reproduction apparatus is used in the home network 
connected via the IEEE1394 , the reproduction apparatus transmits 
the Aligned Unit in a following transmission procedure. A sender 
obtains TF__extra_header from each of the 32 TS packets with EX 
included in the Aligned Unit, and outputs main body of the TS packers 
after decoding based on DTCP standard. When ourpurr ing the TS 
packets, isochronous packets are inserted between any two 
successive TS packets. Insertion points are positions based on 
time indicated by arrivai_tinie_stamp in TP_extra_header . Along 
with the output of the TS packets, the reproduction apparatus 
outputs DTC?_de script or . The DTC? ^descriptor indicates settings 
for copy permission. 3y describing the DTC?_descriptor so as to 
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indicate the copying is prohibited, che TS packs" s are no: recorded 
by other devices when using in the home network connected via the 
IEFF1 3 94 . 

(6) The digital stream in the above emhodimer.c s is che 
AVC lip . However, the digital stream may be a Video Object (V03 ) 
in DVD-Video standard or DVD-Video Recording standard. The V03 
is a I 30/ I EC1 38 18 -I -standard-based program strean obtained by 
mulciplexing the video stream and audio scream. Further, the video 
strearr. in the AVClip may also be based on MPEG4 or WHV standard. 
Moreover, the audio stream may be based on Linear-PC.M, DoIby-AC3 , 
MP3, MPEG-AAC, or DTS standard. 

(7) The movie xn the above embodiments may be obtained 
by encoding analog image signals transmitted via analog 
broadcasting, or may be stream daca const ituced by transport stream 
transmitted via digical broadcasting. 

It is also possible co obcain contents by encoding analog 
or digical image signals that are recorded in a vide oc ape . Further , 
the concents may also be obtained by encoding analog or digital 
image signals chat are directly loaded from a video camera. 
Moreover, concents may be a dccital work delivered by a distributing 
server . 

(6 ) The Graphics Object in the first and second embodiments 
is raster data that is encoded based on run- length limited encoding . 
The run-length limited encoding is adopted for compressing and 
encoding the Graphics Ob j ecu because the run-length limited 
encoding is the mosc appropriate for compressing and decompressing 
the suhcitles. The subtitles have characteristics that a length 
in ahorizontai direction becomes relatively long , and accordingly, 
high compression rate is obtained by using che run-length limited 
encoding. In addicion, the run-length limited encoding is 
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preferable fcr making software for decoding because the load in 
decompression is low. Further, in order to share the apparatus 
structure for decoding between the subtitles and the Graphics 
Object, the same ccnp res si on /decompress ion method as for the 
subtitles is employed for the Graphics Object. However, using 
the run-length limited encoding is not an essential part of the 
present invention, and the Graphics Object may be PNG data. 
Moreover, the Graphics Object is not required to be the raster 
data and .may be vector oata. In addition, the Graphics Object 
may be transparent graphics. 

(9) A target for display effect by the PCS may be the 
graphics for the subtitles selected based on a language setting 
of the reproduction apparatus . Realizing such a display has a 
high utilitarian value, because it becomes possible to realize 
an effect, which is realized by the moving picture itself in the 
■conventional DVD, by the subtitle graphics displayed according 
to the language setting of the reproduction apparatus. 

(1C) A target fcr display effect by the PCS may be the 
graphics for the subtitles selected based on a display setting 
of the reproduction apparatus. Specifically, Graphics for 
various display modes such as a wide-vision, a pan-scan, and a 
letterbox are recorded in the 3D-R0M, and the reproduction 
apparatus select any of the recorded settings based on the setting 
for the TV to which the reproduction apparatus is connected. In 
this case, the display effect based on the PCS is performed to 
the subtitle graphics displayed according to the display setting, 
the subtitles look more impressive and professional . Realizing 
such a display has a high utilitarian value, because it becomes 
possible to realize an effect similar to the effect realized in 
the moving picture itself in the conventional DVD, by the subtitle 
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graphics being displayed according to the display setting of the 
reproduction apparatus . 

(II) In the firs-: embodiment, the Window size is set tc 
be 25% of ar. entire Graphics Plane in order to set the writing 
rate Rc to the Graphics Plane to the rate a: which the clearing 
cf ihe Graphics Plane and redrawing is performed in one frame . 
However, the Rc may be set so that the clearing and re-drawing 
are completed during a vertical retrace period. Given that ihe 
vertical retrace period :s 25% of 1/29.93 seconds , the Rc is IGbps . 
Setting the Rc in such a way has a high utilitarian value, because 
it is oossible to display the graphics smoother. 

Further, it is also possible to perform the writing 
synchronously with line scanning, in addition to the writing during 
the vertical retrace period. By this, it is possible :o display 
the graphics smoother even a~ the writing rate Rc is 256 Mbps . 

( 12 ) In the above embodiments , the Graphics Plane is mounted 
to the reproduction apparatus. However, it is also possible tc 
mount a line buffer for storing decompressed pixels for a line 
in olace of the Graphics Plane to the reproduction apparatus . 
Conversion into image signals is performed by line, and therefore 
the conversion into the image signals may be carried out with the 
line buffer alone. 

(13) In the above embodiment, the explanations are given 
taking the text subtitles for the movie as the examples of the 
graphics . However, the graphics may include such as a combination 
of devices, characters, and colors that constitute a trade marie, 
a national crest , a national flag, a national emblem, a symbol 
and a great seal for supervision or certification that a national 
government uses, a crest, a flag, or an emblem of an international 
organization, or a mark of origin cf a particular item. 
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[14) Zn the firs" embodiment, the Window for rendering the 
subtitles is defined either at an upper side of the screen, or 
the bottom of the screen, assuming chat the subtitles are written 
horizontally. However, the Window may be defined to appear either 
on left or righc sice of the screen so as co display the subtitles 
on the left and right of the screen. In this way, it is possible 
to change next direction and display subtitles vertically. 

(15) The AVClip in the above embodiments constitutes the 
movie. However, the AVClip may also be used for karaoke. In this 
case, the PCS may perform zhe display effect such than the color 
of the subtitles changes along with a song. 

Reference Numbers 

1 3D drive 

2 Read Buffer 

3 PID Filter 

4 TB Buffer 

5 Video Decoder 

6 Video Plane 

7 Audio Decoder 

8 Graphics Plane 

9 CULT unii 

10 adder 

12 Graphics Decoder 

13 Coded Dana Buffer 

14 Stream Graphics Processor 

16 Composition Buffer 

17 Graphics Controller 

2 0G reproduction apparatus 

30 0 TV 
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40C remote controller 

Industrial Applicability 

A recording medium and a reproduction apparatus according 
to the present invention are capanie of displaying, subtitles with 
a display effect . Accordingly, it is pcssiole to add higher values 
to movies supplied in the market , and to activate markets for films 
and consumer products. Thus, the recording medium and the 
reproduction apparatus according to the present invention nave 
high industrial applicability in industry such as the film industry 
and consumer products industry. 
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CLAIMS 

1. A recording medium used for storing data, said recording 
medium comprising: 

a digital stream constituted cy multiplexing a video stream 
and a graphics stream, wherein said video stream represents a moving 
piciure made of a plurality of pictures, and the graphics stream 
includes : 

graphics data representing graphics to be combined with the 
pictures; and 

window information than specifies a window for rendering 
the graphics therein, the window information indicating a widuh, 
a height and a position of the window on a plane, the plane being 
a plane memory of a reproduce ion apparatus that combines uhe 
graphics with the pictures. 

2. A recording medium according ~ c Claim 1, wherein the width 
and height of the window are set so una z a size of the window is 
1/x of the plane, uhe plane corresponding to a size of each picture 
and x being a real number based on a rauio between a window update 
rate and a picture display rate. 

?. A recording medium according to Claim 1, wherein: 

uhe graphics stream includes conuroi information that 

contains crop information specifying a cropping frame within a 

graphics object obtained by decoding the graphics data; and 

the graphics to be rendered in uhe window is a part of the 

graphics object within the cropping frame > 
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4 . A recording medium according to Clairr. 3 , therein the control 
information contains position information specifying a position 
in the window for rendering the part of the graphics object within 
the cropping frame. 

5. A recording medium according to Claim 4, wherein: 

the graphics stream inciudes a plurality cf pieces of control 
information for realizing one of scroll, wipe-in, wipe-out, cut-in, 
and cut-out dispiay effects; and 

each of the pieces of control information includes the crop 
information and the position information thai respectively specify 
a different cropping frame and a position. 

6. A reproduction apparatus used for reproducing a digital 
stream constituted by multiplexing a video stream and a graphics 
stream, said reproduction apparatus comprising: 

a video decoder operable to decode the video stream so as 
to obtain a moving picture made of a plurality cf pictures; 

a graphics decoder operable to render graphics so as to be 
synchronously displayed with the pictures; and 

a plane memory corresponding to a plane and being used for 
combining the graphics with the pictures, wherein: 

the graphics stream includes window information that 
specifies a part of the plane as a window for rendering the graphics 
therein; and 

the rendering of the graphics by said graphics decoder 
includes a clearing of the graphics in the window in saic plane 
memory, and a writing of the graphics to the window in said plane 
memory . 

69 



8/28/2007, EAST Version: 2.1.0.14 



WO 2004/098193 



PCT/JP2004/006074 



7> A reproduction apparatus according to Claim 6, wherein: 
uhe graphics stream includes compressed graphics data; and 
said graphics decoder includes a processor operable to decode 
the compressec graphics data, and a con-roiling unit operable to 
perform the clearing operation and the writing operation. 

8. A reproduction apparatus according to Clair?, 7, wherein: 
a size of the window is set so as to be 1/x of the plane, 

x being a real number based on a ratio between a window update 
race and a display rate of the video scream; and 

the writing operation performed by said controlling unit 
is performed at a transfer rate based on the update rare of the 
window and the size of the window. 

9. A reproGuction apparatus according to Claim 7, wherein: 
said graphics decoder includes an object buffer operable 

to store decompressed graphics data decoded by said processor; 

the graphics scream includes control informal ion char 
contains crop information specifying a cropping frame within a 
graphics obj ect obtainedby decoding the graphics data in the obj ec: 
buffer; 

said controlling unit is operable to crop a part of the 
graphics object within. the cropping frame; and 

the graphics to be synchronously displayed with ihe pictures 
is the part of the graphics object within the cropping frame. 

10. A reproduction apparatus according to Claim 9, wherein: 
the control information contains position information 
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specifying a oosnicr in the window for rendering the part within 
zhe cropping frame; and 

the oar" within the cropping frame is written tc the window 
at the Dosition specified by the position information. 

5 

11. A reproduction apparatus according to Claim 10, wherein: 
the graphics stream includes a plurality of pieces of control 

information; 

the crop pari and the position indicated by the crop 
10 information and the position information respectively in eachpiece 
of control informal icr. are different; and 

said controlling unit is operable tc realize one of scroll, 
wipe- in, wipe-out, cue -in, and cut-cut display effects by 
performing the clearing and writing of zhe graphics based on ihe 
15 crop information and ihe position informal ion in each piece of 
control information. 

12. A reproduction apparatus according to Claim 6, further 
comprising two plane memories constituting a double buff er, wherein 

20 the graphics is displayed by switching the displayed graphics from 
consents stored in one of said plane memories in said double buffer 
to contents stored in the other one of said plane memories. 

13. A method of recording onto a recording medium, said method 
. 25 comprising: 

producing application data; and 

recording the produced application data in the recording 
medium, wherein : 

the application data includes a digital stream constituted 
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cy mult iplexing a video stream and a graphics strean; 

the video stream represents a moving picture made of a 
plurality of pictures, and the graphics strean. includes: 

graphics data representing graphics to be combined with the 
pictures; and 

window information teat specifies a window for rendering 
the graphics therein, the window information indicating a width, 
a height and a position of the window on a plane, the plane being 
a plane memory of a reproduction apparatus that combines the 
graphics with the pictures. 

14.. A program used for enabling a computer to reproduce a digital 
stream constituted by multiplexing a video stream and a graphics 
stream, said program comprising: 

code operable to cause the computer to decode the video stream 
so as to obtain a moving picture made of a plurality of pictures ; 
and 

code operable to cause the computer to render graphics so 
as to be synchronously displayed with the pictures, wherein: 

the graphics stream includes window information that 
specifies a part of the plane as a window for rendering the graphics 
therein; and 

said code operable to cause the computer to render the 
graphics includes code operable to cause the computer to perform 
a clearing of the graphics in the window in a plane memory used 
for combining the graphics with the picture, and a writing of the 
graphics to the window in the plane memory. 

15. A method of reproducing a digital stream constituted by 
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multiolexing a video stream and a graphics stream, said method 
comprising: 

decoding the video stream so as to obtain a moving picture 
made of a plurality of pictures; and 

rendering graphics so as to be synchronously displayed with 
the pictures, wherein: 

the graphics stream includes window information that 
specifies a oart of the plane as a window for rendering the graphics 
-herein; and 

said rendering of the graphics includes a clearing of the 
graphics in the window in a plane memory used for combining the 
graphics with the picture, and a writing of the graphics to the 
window in the plane memory. 

16.- An integrated circuit used for reproducing a digitai stream- 
constituted by multiplexing a video stream and a graphics stream, 
said integrated circuit comprising: 

a video decoder operable to decode the video stream, so as 
to obtain a moving picture made of a plurality of pictures ; 

a graphics decoder operable to render graphics so as to be 
synchronously displayed with the pictures; and 

a plane memory corresponding to a plane and being used for 
combining the graphics with '-the pictures, wherein: 

the graphics stream includes window information that 
specifies a part of the plane as a window for rendering the graphics 
therein; and 

the rendering of the graphics by said graphics decoder 
includes a clearing of the graphics in the window in the plane 
memory, and a writing of the graphics to the window in the plane 
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FIG. 1 
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FIG. 7 A 
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PTS( DSn[PCS)] )>=DTS( DSnfPCS] )+DECODEDURATION( DSn ) 

Where: 

• DECODEDURATION( DSn ) is calculated as follows: 
decode_duration = 0 ; . 

decode_duration += PLANEINIT!ALIZATIONTIME( DSn ) ; 
if( DSn. PCS. num_of_objects == 2 ) 

* decode_duration += WAIT( DSn, DSn. PCS. OBJ[0], decode_duration ) ; 
if ( DSn. PCS. OBJ[0]. window_id == DSn. PCS. OBJ[ 1 J. windowjd ) 

' decode Juration += WAiT( DSn, DSn. PCS. 0BJ[1 ], decode_duration ) ; 

j decode Juration += 90000*( SiZE( DSn. PCS. OBJ[0]. windowjd )//256*l 0 5 ) ; 

else 

1 decodejuration += 90000*( SIZE( DSn. PCS. OBJ[0]. windowjd )//256*l 0 6 ) ; 

decode Juration += WArr( DSn, DSn. PCS. OBJp ], decodejuration ); 
decodejuration += 90000*( SIZE( DSn. PCS. 0BJ[1 ]. windowjd )//256*l 0 6 ) ; 

} 

else if( DSn. PCS. num_6f_objects ==1 ) 

* decode_duration += WAFT( DSn, DSn. PCS. OBJ[0], decode_duration ) ; 

^ decode_duration += 90000*( SIZE( DSn. PCS. OBJ[OJ. window_id )//256*1 0 6 ) ; 

return decodejuration ; 

■ PLANEIN!TIALIZATIONTIME( DSn ) is calculated as follows: 
initialize_duration=0 ; 

if ( DSn. PCS. composition_state= = EPOCH_START ) 

^ ini^alize Juration - 90000*( 8*video_width*video_height//256*1 0 6 ) ; 

else 
{ 

for( i=0 ; i < WDS. num_windows ; i++ ) 

* if( EMPTY(DSn.WDS.WfN[i],DSn ) ) 

initialize Juration += 90000*( SI2E( DSn. WDS. WIIM[i] )//256*1 0 6 ) ; 

} 1 

return initialize Juration ; 

• WAIT( DSn, OBJ, current_duration ) is calculated as follows: 

wait_duration = 0 ; 

if( EXISTS( OBJ. object_is, DSn ) ) 

object_definition_ready_time = PTS( GET( OBJ. object_id. DSn ) ) ; 
current_time = DTS( DSn. PCS )+current_duration ; 
if( current_time < object_definition_ready_time ) 
^ wait Juration += object_definition_ready_time - current Jime ) ; 

return wait_duration ; 18/42 
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